Mining Jobmine: Part 1. Map of the Jobs

First, the map (bigger version here). This map shows where the jobs posted during last weekend's job postings are located. A slightly more interactive version is available here, but takes forever to load.


Yes, there is a job WAY up in the Arctic, and 28 people applied to it. I could have sworn too that there used to be jobs in New Zealand, Hawaii and Australia, but oh well.

A similar map counting the number of applications going to each city is available here (this also takes forever to load). However the two maps look pretty much the same, since the resolution of the bubbles are not that great.

What's also not clear in the maps is the actual number of jobs in Waterloo and Toronto areas. To make it clear, here are the top 10 locations with the most number of job openings.


If we take the top 10 locations with the most number of applications, we get something similar.


Now what happens when we look at the cities that get the most applications per opening? We get something COMPLETELY different.


The more "exotic" places like Saint-Hubert (where the Candian Space Agency is located), California and others don't offer many jobs, but the ones that are offered attract a lot of applications. Note though that I would take the exact ordering in the last chart with a grain of salt for two reasons: (1) the position that actually attracted the most number of applications is offered in "Various Locations" -- take a WILD guess what company offered that position (hint: start's with a "G") and (2) a lot of companies lie about the number of openings they have.

What about the places with the lowest applications per job?


I'll leave you to come up with your own conclusions here.

Finally, a note about data and methodology. Jobmine is the system co-op students/employers at Waterloo use to manage job postings and applications. Job name, location, opening and applications data were pulled off of Jobmine 8am this morning (posting closed 12am last night). Some location names are changed slightly to avoid multiple entries per name, and so that Google's Geomap tool would map it correctly (it mapped "London" to "London, England", etc.). When a job opening was in multiple locations, I took the first one. Jobs that put "Multiple Locations" or "Various" or something ambiguous as their location were deleted.

I plan to squeeze more goodies out of this data, so stay tuned. Incidentally, if you are familiar with Google Charts API and know how to make it go faster, please let me know.

End of Entry

Monty Hall Problem: an intuitive explanation

The Monty Hall Problem is a probability puzzle based on a TV show. Here's the puzzle:

Suppose that you are on a game show, and the host shows you three doors. He tells you that behind two of the doors are goats, and behind one of the doors is a brand new luxury car. He asks you to pick one of the three doors, and if you picked the door with the car behind it, you keep the car. You pick a door (say door #1). The host, knowing which door has the car behind it, walks over to a different door (say door #2) and opens it to reveal a goat. He then offers you to a chance to change your mind (and switch to door #3). Should you make the switch?

If you haven't heard the puzzle a billion times already, think for a bit before reading on.

Here's the answer: you should switch. I'll give two explanations as to why. The first one will (hopefully) appeal to your intuition, and the second one will be an argument using probability.

The Intuitive Explanation

Let's change the game for a bit. Suppose instead of only 3 doors, we have 100 doors: with 99 goats still only one car behind the doors. After you pick a door (say door A), the host opens 98 doors to reveal 98 goats, only leaving one other door (say door B) closed. In this case, would you choose to switch (to door B)? Again think about this first before reading on.

The Probabilistic Explanation

Here's how you might have reasoned about the previous scenario: the only case where switching to door B is not beneficial is when you choose the right door the first time. That only has a 1% chance of happening.

The same reasoning applies to the 3 doors scenario. The only case where switching would not help you is when you choose the door with the car behind it the first time. There's a 33% chance of that happening, and a 66% chance of picking the wrong door. Thus you will double your probability of winning the car if you decide to switch.

End of Entry

logicomix and atlas shrugged

Though Logicomix is in no way historically accurate, it portrays well a feeling that I think a lot of us share. I'm sure that a lot of people saw the delicate beauty in Euclid's Elements, in the (relative?) certainty of mathematics, and hoped that something remotely similar to Elements can be made to solve dilemmas that come up in every day life--a consistent philosophical system deduced from the basic facts that everyone would understand and agree upon.

I had the naiive thought once. It's comforting to believe that there's a way to make a limited set of assumptions about life and existence, and derive from those a consistent set of beliefs about everything from metaphysics to ethics to politics.

From what other people are saying around the interweb, I'm not sure if Bertrand Russel would be the right person to attribute this set of feelings to. Actually, this might sound weird, but a more appropriate person would be Ayn Rand.

I recall in Atlas Shrugged, she speaks of people -- even philosophers (and logicians?) -- using logic to prove that logic is flawed/inadequate. These people were, of course, the "bad guys", the people that are held by the masses to be the "leader" of their chosen fields, but who are really there to foil the heros, the proponents of logic.

I loath hearing things about so and so "used logic to prove that logic is inadequate". I heard it once on numb3rs, too, so it's quite annoying. I had no idea that this in fact had been done. It's Gödel's Incompleteness Theorem. Of course!

Did Ayn Rand know about Gödel? Would that changed her mind in any way? If we make the assumptions that (1) human life/experience/society is much more complicated than the natural numbers, and (2) a system of philosophy (built up from a few axioms, for our purpose) can be thought of as a model of the world, then Gödel seems to imply that any of our philosophical system is either incomplete or inconsistent.

I'm probably missing something very important here.

End of Entry

Moment of Fame

I celebrated Chinese New Year this year by writing this article for the Facebook Data Team:

http://www.facebook.com/note.php?note_id=304457453858&id=8394258414



End of Entry

The Impostor Syndrome

The Impostor Syndrome is a phenomenon where a person is incapable of internalizing success, often attributing it to luck, good timing, or pure fluke. The person is fearful that they might be "found out" to be less intelligent or competent than people make them to seem.

It's interesting that this phenomenon is prevalent enough to be both studied and given a name. Very interesting.

(And yes, I've felt this way many times, but it also seems that I am capable of internalizing success. Or maybe I'm just giving myself credit for effort, in which case I should remind you of this quote.)

End of Entry