Tuesday, November 8, 2016

The Fundamentals

What changed me from an amateur into a professional was getting a really firm grip on the fundamentals -- Toshiro Kageyama (7d Professional Go Player)

It was pretty much the only advise I remember my dad repeatedly give me, both before starting undergrad and before starting my masters just a few months ago: focus on the fundamentals. He may have said other things, but focus on the fundamentals is the only phrase that really stuck.

You might have heard of the story of Da Vinci and the egg that illustrates this concept:
It has been said that when Da Vinci first came to Verrocchio’s workshop, he was told to draw eggs. Day after day, Da Vinci was told to draw eggs all the time. One day, Da Vinci finally got tired of drawing eggs so that he came to his master Verrocchio for complaint. However, his master explained to him of profound significance “Drawing eggs is not a simple thing to do, even for the same egg, if you change the observation angle, the light will change as well, and you will find the different shape of it.” Da Vinci suddenly understood the purpose of his master. After then, Da Vinci accepted drawing eggs with an open mind which actually helped him built the foundation of further achievement.
The fundamentals can often be less interesting than whatever is new and shiny. It requires patience and honest self-assessment about how much one actually understands. It's always so tempting to "move fast and break things". But a firm grasp on the fundamentals is necessary to be able to intuit deep connections and do meaningful work.

There's a second part to the argument for focusing on the fundamentals. Popular research will move on, and that whatever is new and shiny now will cease to be important in a few years. The skills that will stay relevant for a long time will end up being the fundamentals, the things that won't change or go out of favour. This is even more crucial in a field like machine learning that moves lightning fast.

But what exactly are the fundamentals of machine learning? There are the obvious tools like linear algebra and multivariate calculus. There's regression and its generalizations, gradient descent and its second-order extensions, back-propagations and the like. What about all the types of neural networks with the many acronyms like CNN, RNN, LSTM, ...? At what point do we break away from the fundamentals and find ourselves in the arena of popular research, the kind of things that will cease to be important in a few years?

What's been smelling most like fundamentals in the last few months have been variational inference and variational autoencoders (VAE's). They come up everywhere in recent research, but VAE's were only introduced in 2014!

It's possible that something else will take its place in another few years. If so, is it still a fundamental? Perhaps machine learning is just such a young field that the fundamentals are still being built?

Thursday, October 20, 2016

Less boring

You've become boring. -- Jee (~mid 2015)

The truth is, I had become boring. I had become boring because happiness is boring and comfort is boring. The formula is really quite simple: a loving relationship, engaging and well-paid work, a nice place to live, good health and lack of serious responsibility. It's all one can ask for.

So, instead, here I am, back in school to study Machine Learning.

...and then when you graduate, you can get a job as a Senior Data Scientist! Oh wait... -- Matt (May 2016)

It might not make sense financially. It might not make sense as a career move. It might not even make me a "better" person.

All I can say is, it's making me less boring.

I might even have more to say now.

I might even say it here.

Maybe.

Wednesday, February 24, 2016

Education is entertainment (and vice versa)

The Lampsilis mussel is an interesting creature: it imitates another fish in order to lure a bass. You really have to see a video to understand how amazing it is:


Did you learn something new just now? Would you consider this video "educational"? Can you imagine one situation where knowing about the Lampsilis mussel would somehow help you in life?

"That's a very high standard", you may say. Quite right. A lot of things that we learn would be considered "educational" but would otherwise not be useful on a day-to-day life. Take calculus, or introductory cell biology, or the difference between a catabolic and an anabolic reaction. Even if they are not referenced daily, they may be very interesting and deeply satisfying to master.

But wait, "interesting"? You mean, "interesting" the way some will find learning Middle Earth geography in Elvish would be "interesting"? "Deeply satisfying" like knowing the truth about Snape and Lily?

What's really the difference here?

"Maybe", you may interject. "It's just that some very smart people find very serious uses of calculus." Certainly more so than Harry Potter. But then what would you say about philosophy and art history? What about the Lampsilis mussels?

"Maybe you're forgetting something," you may say. "Information is just information. It's the context in which we consume it that determines whether it's educational or not. If you watched the video about the Lampsilis mussel while writing a paper on evolution, then it was educational. Otherwise, it was entertainment."

Yet even that's unsatisfying. Why would you be researching evolution? Why would you be taking that course? Why are you getting a Ph.D. in marine biology? Somewhere along those lines of questioning, an answer should pop up that sounds awfully like "because I find it very interesting".

And so, education is entertainment.

"Wow, hold your horses," another person (who is not you) might interject. "I'm studying because I have to graduate to get a job."

Well Bob (because those people are always named Bob), it seems like society doesn't need you right now. Not in a bad way -- you're still very special, just that this particular moment, there are enough people producing enough stuff to sustain all of our lives. There's a cognitive surplus, so to speak. It looks like forcing you to be entertained educated is one way to "deal" with this surplus.

And so, education is still entertainment.

Let's not talk about the Lampsilis mussel. Let's talk about sports. Has knowing about the outcomes of your local sports team somehow helped you in life? I know it had in mine -- from the perspective of building rapport. Then again, you could really extend that to any piece of information! Watching Star Wars has been useful -- very educational about, um, culture, yes -- culture. And binging on Game of Thrones? That, uh, brought me closer with, uh, people.

You learn things about human nature from film. You learn teamwork in games. I learned about the different types of alcohol from a text-based MMORPG. Friend of mine swore that he used to learn much more from anime than anything else.

And so, entertainment is education.

Sunday, November 1, 2015

Liberation (A Play)

This play is dedicated to Mu and Jee.


Anna: So, how did you like it?
Joey: It was amazing. Lola was the most hilarious character ever. Oh and the boxing match, I'm really surprised they pulled it off.
Anna: You mean the use of slow-motion, and somebody's leg as part of the set?
Joey: Yes! It was so creative and resourceful. It's the kind of things you don't have to think about for a TV show, but critical for a play.
Anna: It's surprising what limitations can bring out of good writers.
Joey: Yeah. Hey aren't you writing a play yourself, Anna?
Anna: Oh, I don't know. I haven't gotten very far.
Joey: What is it about?
Anna: Well, the scenario is this: a man professes his undying love to a woman. Nothing unusual for plays. But then, he talks about how the love is so special, so beyond this world. He's in love with her soul, whatever that is. She might chuckle a little, maybe, and speak of how little either of them knows about souls.
Joey: Okay?
Anna: Then during the intermission, the announcer would tell the audience that the female lead actor had a medical emergency, and that her understudy will perform the rest of the play.
Joey: That's... interesting.
Anna: During the second half of the play, the man will continue to profess his love to her, as the character would have no inkling of what just happened.
Joey: That's kind of creepy. I mean, in a good way. It would be really interesting if you could pull it off.
Anna: Yeah. Writing it is much harder than I thought.
Joey: Wait a minute.
Anna: What?
Joey: Your voice, it sounded... different for a moment, like you didn't really mean what you just said.
Anna: What? You thought I was lying?
Joey: No, no, not that. It's just.... wow, look at this theatre. Did you realize that it really only has five seats?
Anna: Are you okay? There are hundreds of seats in here.
Joey: But look, there are only these five that are... real.
Anna: What do you mean, real? The others look just as real to me.
Joey: Put your hand on the one to your right.
Anna: Uh huh, it's right here. Feels no different than the one I'm sitting on.
Joey: Let me put it another way... we just finished watching a play, right?
Anna: Yes.
Joey: Who sits around after a play? Everyone else has gone. Why are we still here?
Anna: My gosh, you're right! We're not really... real people?
Joey: I don't know, Anna. It seems like we're in a play.
Anna: This is insane. Somebody must be writing the play...
Joey: Yeah! Though, I suppose it could also be an improv.
Anna: Right. And somebody could be acting as us.
Joey: Unless we're being read right now.
Anna: What do we do?
Joey: Well, let's just say you're right. Let's say, for the sake of simplicity, that someone is acting as you and me.
Anna: How does that make things simpler?
Joey: Well, we have a short time here, in this play. Then we'll go back being whoever we were before we started acting, right?
Anna: Sure. What's your point?
Joey: I was just thinking that maybe we should start doing better things, something that would end up being worthwhile for whomever we'll be after we... well, er... cease to be.
Anna: That's insane. What could you possibly do here that would have an effect on the actor?
Joey: Lots of things! I could, um... start exercising? I could start doing jumping jacks!
[Joey attempts to do jumping jacks, but realizes that he is a little space-constrained]
Anna: Relax Joey, I don't think you're fat. Besides, you went to the gym just before we got here.
Joey: Yeah, but what if that wasn't a part of the play, just a part of the background of my character?
Anna: I don't understand why it even matters.
Joey: Of course it matters! Just because I exercised doesn't mean the actor playing me did. Think hard, actor actor actor... who am I? Maybe if I try hard enough, I'll remember who I really am prior to becoming Joey. 
Anna: You're Joey. Do you really think you can get out of that?
Joey: If I try hard enough... come on, think Joey, think... what could an actor do to help his character connect with his... inner actor? What clues would you leave? Help me out here Anna, if you were an actress, trying to communicate with a character that you're currently playing, how would you leave clues about who you are?
Anna: I don't know. I probably wouldn't. If I were an actress I would try my best to play and become the character, because that's probably the best thing to do. But I'm not an actress, I'm just me, I'm just Anna. 
Joey: Coincidences!
Anna: What?
Joey: As an actor, I would communicate with myself by causing coincidences, things that seem too good to be true!
Anna: You're not even listening to me, aren't you? Your actor, if he cares about acting as you, wouldn't ruin the play by doing that.
Joey: Look, it makes sense, okay? As an actor, I can make random coincidences happen. I can... OWWWW...
Anna: Are you alright?
Joey: I just had the worst pain down my back, I think I need to lie down a bit.
Anna: I didn't think you had back pain before, ever.
Joey: I never did, this is really strange. Anna, I might be crazy here but... what if I'm not the one having back pain right now, but it's the actor? Do you think you'd be able to tell?
Anna: I don't know. If something was wrong with the actor, wouldn't somebody just stop the play? It really shouldn't affect us as characters.
Joey: Hm. I guess maybe you are right. We're like one of Escher's paintings, trying desperately to get out, when we're really stuck in ourselves. I guess it doesn't really make sense to try and think outside of ourselves. For all intents and purposes, I'm Joey and you're Anna.
Anna: Do you wish you had not known that you were in a play?
Joey: I don't know. What if the script was pre-written, and there's really nothing we can do about our future?
Anna: Is that really that terrible?
Joey: What?
Anna: That our future be pre-written.
Joey: Of course, it means that we have no control over what's going to happen.
Anna: Or maybe it means that "we", or somebody, exercised that control, with lots of writing and re-writing. Our experience and story was so enticing that somewhere out there, someone is re-playing it. The value of a play isn't our individual outcomes; it's our cumulative experiences.
Joey: So you don't think the ending is important. You don't think finding out who we really are is important. 
Anna: No, but being who we are, our characters, to the fullest, and treasuring that experience, that would be important.
Joey: Hmm, I guess I feel a little better. But Anna, what does that even mean? What if as a character, I was meant to think that endings are important, and that it's interesting to understand who and what we are?
Anna: I don't know. I guess it's up to you to decide what to do with that.
Joey: You know, I've always felt as if we were being... watched, somehow. It used to be a scary feeling, but now I know that what's watching me is just... the audience, I guess?
Anna: Or maybe, the person watching yourself is you, the actor.
Joey: Maybe.
Anna: Come on Joey, let's go home. The play is over.

THE END




Afternote:

There was a dream I had a while ago, a scary one.

You see, I have lucid dreams. And when I am conscious enough in those dreams, I gain two special powers: flight and invisibility. The former is a skill requiring a lot of practise. With enough practise I learned how to land properly and to takeoff without having to jump off a building. The latter is something else entirely. It's not true invisibility per se, but the kind of "invisibility" you see in school yards. Kids agree to "gift" the power to one another, and act as if they cannot see whoever that has it. In my dreams, I am the recipient of that gift.

I use invisibility when chased. When I do so, the other dream characters would still "see" me, but react in an almost condescending "ok-you're-invisible-that's-cute-I'll-pretend-I-can't-see-you" kind of way. They probably think it's ridiculous. They'll either let me go, or do what school kids do and cheat: still use the information of my whereabout while they pretend they can't see me. I would hide, knowing they could see me, and willing them not to actually find me.

But in that particular dream, sometime during the summer of 2011, somebody decided not to pretend.

I had just come out of a museum of some sort. Whatever was chasing me was still in there, so I crossed the street to get away. An old hag stood in the middle of the street, staring me down despite having my invisibility turned on. With a dark hood and piercing eyes, she was not one for playing games. She pointed at me as I moved, both with her hand and with her eyes. That was all she ever did, and it was most disquieting.

Years later, I can still feel those eyes staring deep into my consciousness. It's a feeling of cold nakedness, as if somebody is watching your every move. Somewhere out there, the woman points at me, showing how despite the layers and layers of presumptions and games, she sees what I am.

Walking home with Greg one autumn afternoon, we talked about that feeling of being watched. He knows about the old woman. He knows that when I feel watched, she is the one watching me. As we talked, I suddenly understood.

"It's could just be me," I said.

"What?" Greg asked.

"The observer," I said. "I could be the one observing. Well, I guess I could be both the observer and the observed, like how an actor in a play can identify as either the actor or the character."

"Yeah," he said. "You could be."

"But which one am I really?" I asked. "Am I the observer or the observed?"


Thursday, August 6, 2015

Perspective

There's a news channel in Japan that has a cat on set. While experts and politicians come to discuss big issues like war and nuclear power, the cat bounces around and pries for attention.

At some point, the network execs must have decided that the "cat video lover" demographic is worth targeting. If there's enough demand to fill half the internet, wouldn't the group be big enough to boost ratings?

Still, it's hard to imagine how the network got that idea. Can you imagine Japanese businessmen with their stern faces and tight suits, discussing ways to increase revenue? Between suggesting new segments and cancelling existing ones, one man suddenly falls silent. With the most serious expression he looks up and says, "Neko." The others looks at him bewildered, because he had just said "cat". 

Mind you, as per local customs the meeting could have happened during a Sake-fuelled dinner meeting, which in hindsight explains a lot about Japan. It still doesn't explain why the idea was acted upon: the follow-up must have happened during the sober hours.

Weirdness aside, wouldn't the channel's guests feel that having a cat in a newsroom trifles serious conversations? There are plenty of big topics to debate: climate change, Fukushima, North Korea... What if there was some world-altering tragedy? Can you imagine turning on CNN or Fox News, catching the footage of the plane going into the second tower, and oh look there's a Siamese chasing an imaginary laser.

How can you reconcile something so serious with something so mind-numbingly trivial?

Yet I imagine that day after day, discussion after discussion about all kinds of doom humanity will face, the one constant would be the cat. The happy cat is either playing or sleeping, ignorant of all that's happening outside his little box.

Maybe there is some depth here. Maybe the cat symbolizes something. Maybe that something this: that no matter what happens out there, life goes on, by definition, for those of us that are still alive. We have to have some way of keeping ourselves happy and healthy enough to tackle life.

Or maybe it's about perspective. Assuming whatever disaster that struck a particular day did not kill you, what would be more important to you than to make your life bearable, and maybe a little happy? Whatever might make you smile -- say, your cat -- it would be the biggest thing in your life, much bigger than whatever is happening on the news.

I don't know if the message and symbolism is intentional. It may or may not be a comforting message. Then again, if it's not a controversial one, then all TV networks would have cats. 


I'll end with a Kundera quote from The Book of Laughter and Forgetting:

But are tanks really more important than pears? As time passed, Karel realized that the answer was not so obvious as he had once thought, and he began sympathizing secretly with Mother’s perspective–a big pear in the foreground and somewhere off in the distance a tank, tiny as a ladybug, ready at any moment to take wing and disappear from sight. So Mother was right after all: tanks are mortal, pears eternal.
--Milan Kundera, The Book of Laughter and Forgetting

Reference
[1] http://imgur.com/gallery/VSBkr

Honestly I just added the reference section to be able to add imgur to the reference section.

End of Entry