13 Jun 2016
My general confidence this week.
This week we studied various methods of linear and logistic regression. We went pretty deep into the mathematical underpinnings for why these techniques work, much further than I had gone in undergraduate statistics or graduate school work. The way we studied these regression techniques was to build our own models from scratch and compare their performance and outputs with that of Python’s own built-in libraries: statsmodels and scikit-learn. I am definitely not a fan of the documentation for either or these modules (there is NO reason that technical writing need be inaccessible) but scikit-learn at least seems slightly more intuitive. Both programming libraries are however very powerful and I was impressed with the speed with which they both were able to fit complex models on our datasets.
Being able to build a model to represent data is of no use to anyone unless you are able to interpret what the model actually means and how statistically significant it’s results are. We therefore spent a good deal of time this week learning various ways verify a model’s validity. This is something I spent a good deal of time doing during my year’s of experimental work in labs so I felt pretty good working through these assignments.
What made this week particularly hard was my < 4 year old computer decided it would be a great time to die. It did this right in the middle of a quiz. The screen flashed shades of green and blue, it went totally black and then I heard a series of beeps coming from the inside the computer (rather than the speakers). Some sleuthing seemed to indicate that the RAM could potentially be bad. I purchased new RAM and a new solid-state drive on Amazon. Wednesday, after a long day at Galvanize, I spent the entire evening (until roughly 2am) installing the new parts and installing all software I needed.
Seemed like everything was running smoothy during the morning, but alas, even with the new parts, the computer continued to crash. So…I am now writing to you from a brand-new macbook. I have 14 days to see if I can fix my old one and return this new one, otherwise I’ll be keeping my new computer. It is a good 5 pounds lighter than my previous, so that’s nice.
La fin du semaine
The week ended with a difficult assessment covering math, statistics, python, pandas, numpy, SQL and more. My sister flew in Friday evening and we have been enjoying a weekend of hiking, celebrating my cousin’s PhD and eating far too much food.
Next Week: The real meat of the course begins – Machine Learning.
06 Jun 2016
Today, the high in Seattle hit 92°F. It’s very hot in my apartment, I feel like a normal distribution with an increasing standard deviation…
Animated normal distribution with a changing standard deviation. Made with matplotlib and imageMagick.
Before entering the Data Science Intensive Program at Galvanize, I reached out to current and previous cohort members to see if they had any thoughts or advice for someone thinking to enter. Nearly every person came back with a glowing review and the same analogy:
> "...it's like trying to drink from a firehose."
They weren’t joking. There is so much information being thrown at us at one time that it is physically impossible to absorb it all. The best you can do is take good notes and try to take in as much as you can.
In this week we covered an incredible amount of material. We covered (no joke) about a year’s worth of lectures on probability, statistics (frequentist and bayesian), A/B testing, hypothesis testing, and bootstrapping all in one week. Because of memorial day, all of that was crammed into four days. Needless to say, I’m exhausted, but satisfied.
Each day’s lectures were complimented with programming exercises illustrating the topics of the day. For the A/B testing exercise for example, we used data from Etsy to determine if changing their homepage would drive additional customer conversion. For the Bayesian lecture, we developed and performed simulations for coin flips and die rolls to illustrate the concepts behind Bayesian probability. This topic was somewhat mind-blowing to me, as Bayesian probability is a way of thinking about statistics which is totally different and foreign to the ways that most people (including myself) are taught.
This was also the second week of pair-programming and I’m finding I like it more and more. Pair-programming is when one person “drives” while the other person “worries”. You switch off every 30 minutes or so. The brilliance of pair-programming is that, in addition to learning to work with other people, by the end of the day, we are often very tired and having a partner helps. Working in groups makes you answerable to your partner. Your mutual success depends on both of you working hard to get the assignment done. We switch partners daily, so each day has a different dynamic. Sometimes I’m the stronger programmer, sometimes I’m not. I’ve found it’s a nice way of humbling oneself.
Thanks for reading.
Next week: It’s Linear and Logistic Regression, sounds fun huh?
29 May 2016
Wow, what a week!
If I thought the first week was tough, I was wrong. I haven’t worked this hard in a long time. It’s incredibly exciting though to be working and learning in a place like Galvanize. My fellow classmates come from all walks of life: structural engineers, web developers, business analysts, even a snowboarding instructor. The week started off with a two-hour assessment on Python, Numpy, Pandas (not the bear), SQL, Calculus, Linear Algebra, Probability, and Statistics. I’m very glad that I took Week 0 because I know for a fact that if I hadn’t, the test would have been much harder. This test will serve as a baseline for our progress going through the program.
After the initial assessment, we went through all the nitty-gritty, boiler-plate at Galvanize. We got our keys, wifi setups, learned the rules. Turns out that after the program ends in August, I get 6-months of access to all the facilities that Galvanize has: conference rooms, desks, the roof deck, social events, networking, etc. That’s an awesome bonus I was unaware of.
Going forward, the schedule more or less follows the table below.
|8:30 - 9 am
|9 - 11ish
|11ish - 12ish
||Individual Programming Assignment
|12ish - 1:15 pm
|1:15 - 3ish
|3ish - 5pm +/- 30
||Pair Programming Assignment
EDIT: The reality seems to be that I leave Galvanize around 6pm or later.
We covered far too much information this week in lectures to go over on this blog, but here are the highlights.
Git + GitHub
One of the biggest focuses of this week has been getting familiar with Git and Github. These two tools are fast becoming the industry-standards for version control. They allow scientists, engineers, hobbyists and the like to coordinate projects from all over the world without writing over each other’s changes. In addition, if you were to say, write a line of code that breaks everything, git contains a history of what’s called “commits”. You can revert to previous commits and get back to your working version. Git, is the program which runs version control. Github, is an online service similar to dropbox that allows you to host and collaborate with others. Here’s a link to mine. There isn’t much there yet but it will be filling up fast.
SQL (it’s just a puzzle to get stuff)
In the era of big data, sometimes the biggest problem is just accessing the information you need and leaving the rest behind. SQL (Structured-Query-Language) is a language used by many industries to access their data. Here’s a little example. Let’s say, I have a database called “my_table” and it contains a “favorite_cheese” column.
SELECT * FROM my_table WHERE favorite_cheese='camembert';
This query would return a table of every row in the table ‘my_table’ where the ‘favorite_cheese’ column was equal to ‘camembert’. Seems simple enough but by being creative you can perform incredibly complex operations to access results which are just what you are looking for.
We also covered Bash, Object-Oriented Programming, Pandas, AWS and more but I’ll try to address those in future posts. The one thing I will mention is that if you type
into your shell, you’ll see an ASCII version of Cathy saying “Chocolate, Chocolate, Chocolate, AACK!”. How useful is that?!
The week ended with a happy hour hosted specifically for Galvanize’s Data Science students past and present. We were able to meet students from the previous cohort and learn about their experiences during and after Galvanize. We’ve got a great group and I’m happy to be learning and working with these people.
Next Week: In Week 2, the focus will be on statistics and probability. Stay tuned!
23 May 2016
View of pioneer square from Galvanize’s headquarters in Downtown Seattle.
In the data science program at Galvanize, you sign up for a 13 week, intensive course in Python, machine learning, statistics and more. It is meant to be a highly efficient means of transitioning into the data science and analytics field; a transition I’ve been excited to make for some time now.
It turns out that Galvanize offers a Week 0, voluntary week, specifically focused on getting the members of the cohort up to snuff when it comes to python programming and linear algebra.
As I knew, going into the program, Galvanize was going to be an intense academic challenge. Already on Day 1 of week 0, I was having to work quite hard, thinking back to my undergraduate days when I worked with vector spaces and matrix algebra. Luckily, nothing was too taxing as of yet.
I’ve been enjoying playing around with the atom text editor which is a very powerful and flexible way of writing in many different languages. In fact, I’m writing this entry using markdown right now. One of my favorite things about it is the fact that I can use LaTeX math notation right in the editor meaning I can write out complex equations, arrays and the like quite easily using the text editor interface.
The location and setting of Galvanize are both quite awesome. It is located in the heart of Seattle’s Pioneer Square in an renovated brick building (which apparently used to be NBBJ’s headquarters). Housed in the building, in addition to the Galvanize’s education programs are many startups, making the atmosphere busy and exciting. Because this week is voluntary, only part of my future cohort is here, but so far everyone, including the teachers seem very intelligent, motivated, and friendly.
I’m excited for the next 14-ish weeks of my life and the challenges and opportunities that this fellowship will bring me. My plan is to write a blog entry for each week of the program so people can track my progress, and see what a programming bootcamp is really like.