Gil's Tech Blog

Summly acquired for $30M, but the technology wasn't even theirs?

Tue, 26 Mar 2013 17:19:20 -0400

Summly acquired for $30M, but the technology wasn't even theirs? :

The major news outlets led me to believe Summly had invented their own NLP technology. Apparently not. So what exactly did Yahoo buy? Ouch.

Need developers, designers and creatives to grow your business? Meet CollabFinder.

Tue, 13 Nov 2012 13:07:30 -0500

Need developers, designers and creatives to grow your business? Meet CollabFinder.:

Fascinating interview with my partner Saha on how he came up with the idea for CollabFinder. I especially love the photo from Jelly, circa 2007. Everyone in that room went on to do great things.

I am a Terrible Programmer

Thu, 08 Nov 2012 19:18:33 -0500

I am a Terrible Programmer:

Just because you graduated with a degree in computer science from Penn does not make you the next Jason Cohen or Patrick McKenzie. You have to do more than just finish your homework to become a good coder.

Like most things in life, the answer to what a good coder is, is somewhere in between the guy who wants to get it out fast and the guy who wants to make it beautiful.

The answer is: a good coder knows when something should be quick and dirty, and when something should be thorough and clean. You learn to ask: is this really necessary? And sometimes taking a extra few hours to plan out how you’re going to build something is necessary.

When Relational Isn't Enough: Neo4j at Squidoo

Thu, 25 Oct 2012 16:54:11 -0400

A look at how Squidoo used the Neo4j graph database to power our new Postcards product.

jordan orelli: Why I went from Python to Go (and not node.js)

Fri, 14 Sep 2012 19:44:50 -0400

jordan orelli: Why I went from Python to Go (and not node.js):

jordanorelli:

People often ask me why I have decided that I’d be writing the bulk of my new code in Go, which I started programming in November of 2011 while attending Hacker School. At that time, concurrency was a very hot topic in Hacker School, and we were all trying out different ways of writing…

Now I want to try Go…

How Stripe secured the servers that ran their hacker Capture the Flag contest

Mon, 27 Aug 2012 14:47:17 -0400

How Stripe secured the servers that ran their hacker Capture the Flag contest

I love that more and more companies are taking advantage of the cloud to minimize RTT for end users

Tue, 14 Aug 2012 15:42:20 -0400

I love that more and more companies are taking advantage of the cloud to minimize RTT for end users

If you use Gmail

Tue, 07 Aug 2012 00:29:00 -0400

Stop what you’re doing and enable Two-factor authentication right now, so this will never happen to you.

Really digging this operations kung fu

Tue, 31 Jul 2012 12:08:09 -0400

A 3scale engineer discusses the curious case of a redis server that frequently EOMs, ultimately uncovering a cross-provider network problem.

What Powers Instagram: Hundreds of Instances, Dozens of Technologies

Tue, 10 Jul 2012 11:47:43 -0400

instagram-engineering:

One of the questions we always get asked at meet-ups and conversations with other engineers is, “what’s your stack?” We thought it would be fun to give a sense of all the systems that power Instagram, at a high-level; you can look forward to more in-depth descriptions of some of these systems in the future.

While the stack is about what I’d expect from Instagram, I found the conversational format of the post to be particularly useful for describing architecture.

Chatting with a hacker while debugging his trojan virus

Tue, 19 Jun 2012 12:19:50 -0400

Chatting with a hacker while debugging his trojan virus:

I probably would have just asked him how the weather’s been.

The Story of Apple's Switch to Intel CPUs

Wed, 13 Jun 2012 18:16:15 -0400

The year is 2000. My husband (JK) has been working at Apple for 13 years. Our son is a year old, and we want to move back to the East Coast to live near our parents. To do this, my husband will need to be granted permission to telecommute. This means he can’t be working on a team project and needs to find something independent to do.

The plan to move is a long-range plan. JK lays the groundwork early to start splitting his time between his Apple office and his home office. [By 2002, he is working at home full-time in California.]

He sends mail to his boss who, coincidentally, was my husband’s first hire when he started at Apple in 1987:

Date: Tue, 20 Jun 2000 10:31:04 -0700 (PDT)
From: John Kullmann <jk@apple.com>
To: Joe Sokol
Subject: intel

i’d like to discuss the possibility of me becoming
responsible for an intel version of MacOS X.

whether that’s just as an engineer, or as a project/
technical lead with another person - whatever.

i’ve been working on the intel platform for the last
week getting continuations working, i’ve found it
interesting and enjoyable, and, if this (an intel
version) is something that could be important to us i’d
like to discuss working on it full-time.

jk

Apple Inc: How does Apple keep secrets so well? via Quora.

What an amazing story.

"The main takeaway that I have been able to synthesize from all of this data is this: Greatness..."

Mon, 04 Jun 2012 10:19:59 -0400

“

The main takeaway that I have been able to synthesize from all of this data is this: Greatness always comes from someone with a finely honed craft, a craft honed to the point of muscle memory. In baseball, you can’t be thinking about which hand goes where on the bat, and how wide your stance is, and where your feet are placed if you want to hit a fastball. All of those decisions have to be muscle memory, and you must have a clear head that is simply thinking about “showing up to play.”

Similarly, in software, you can’t be thinking about which programming language you are using, and whether you are using MongoDB or MySQL, or whether photogrid layouts are the hot new thing or not. You will never hit the proverbial fastball if that is the sort of junk filling your head. Rather, creating and shipping products needs to be muscle memory. You just need to have clear eyes, a full heart, and be ready to show up and play.

”

- Oh, the Places You’ll Go! by Dalton Caldwell.

Software engineers don't need to know advanced math, do they?

Sat, 02 Jun 2012 15:32:18 -0400

This is the best argument I’ve ever heard for why advanced math training is useful for software engineers.

Spoiler alert: it’s not about recalling facts, but stretching your brain so it can work comfortably with abstractions.

According to the “filling a vessel” view, education consists largely of pouring facts into our brains, and using what we have learned consists of pouring it back out. That is, dare I say it, a highly simplistic — and erroneous — view of education. But it’s one that the education establishment (which I’m in) fosters every time it offers a course and then measures the results by setting a largely regurgitative, three-hour, written exam.

In contrast, all the evidence from several decades of research both into the way the brain works and into the learning process — and there is masses of such evidence — says that the acquisition of facts and algorithmic procedures are merely surface manifestations of what goes on when people learn. (We know they are surface phenomena since we generally forget them soon after the last exam is over.) The real value of education is something else. Our brains are perhaps the world’s best examples of an adaptive system. When we subject the human brain to an extended educational experience, it undergoes permanent changes. In physical terms, those changes are the growth and strengthening of certain neural pathways. In functional and experiential terms, we acquire new knowledge and skills.

-Do software engineers need mathematics?

Latency numbers every programmer should know

Thu, 31 May 2012 08:57:40 -0400

https://gist.github.com/2841832

I’ll admit I have never worked in nanoseconds, and I do not have any of these latency numbers memorized. Makes a good reference though, as back of the envelope latency calculates do come in handy.

Google's New BigTable + Relational Database

Thu, 31 May 2012 01:04:53 -0400

Many of the services that are critical to Google’s ad business have historically been backed by MySQL. We have recently migrated several of these services to F1, a new RDBMS developed at Google. F1 implements rich relational database features, including a strictly enforced schema, a powerful parallel SQL query engine, general transactions, change tracking and notiﬁcation, and indexing, and is built on top of a highly distributed storage system that scales on standard hardware in Google data centers. The store is dynamically sharded, supports transactionally-consistent replication across data centers, and is able to handle data center outages without data loss.

Google’s got a new database that combines both BigTable and relational db features. The PDF is remarkably easy to read - don’t be shy.

Tell me more about that URL

Fri, 11 May 2012 17:43:07 -0400

Tell me more about that URL:

We’re working on a new project at Squidoo that requires extracting as much metadata as possible from any given URL. As part of the project, I’ve created a simple open source PHP library called php-url-meta that crawls a URL and returns an object containing the page title, description, keywords, author, and a thumbnail image.

The library favors Open Graph tags in most cases, falling back to standard meta tags if OG tags are not present. Author data comes from rel=”author” markup. Needless to say, because markup is totally up to the entity who created the URL, there are no guarantees that anything will be available.

AWS is NOT the prime enabler of scalability

Thu, 10 May 2012 13:05:07 -0400

Lately I’ve read a lot of articles proclaiming that scalability is no longer an issue thanks primarily to AWS. As a devops engineer who lives and breathes this stuff, I’d like to point out that there are oodles of other technology advances that are more critical for scalability than simply being able to spin up virtual servers on demand.

The simplest possible example of why more servers != scalability is that of a MySQL query. If you run an unindexed query on a large table, you can add more slaves all day long but you still aren’t going to be able to service requests more quickly. Add an index, and suddenly you can service hundreds or thousands of similar queries with the same amount of resources as it took to run a single unindexed query.

I’d argue that the prime enablers of web scalability are:

Cheap RAM. Storing a dataset in memory improves performance, which means more queries can be served in less time. Servers with 80GB of RAM (or more) are not only possible, but common.
Non-relational databases. Everything gets measured these days, but relational dbs aren’t up to the challenge of storing large amounts of analytics data. Of course there have always been non-relational dbs, but the proliferation of Cassandra, Mongo, and other key/value stores enable us to store massive amounts of data concurrently with single digit millisecond response times.
Content Delivery Networks. Most web requests are for static assets like images, javascript, and CSS. Gone are the dark days of serving these assets using resource intensive processes like Apache. Now our app servers are free to serve exactly that - the application.
Better libraries & documentation. Resources like Cal Henderson’s Building Scalable Web Sites brought modern concepts like memcache and asynchronous queues to the masses. Now documentation is ubiquitous and all web languages offer multiple excellent libraries enabling scalable web application development.

You might argue that AWS offers many of the services I’ve described above. It’s true. But AWS was not the first to offer them, nor is AWS the only (or even cheapest) option today.

The Anatomy of Blekko's Search Database

Wed, 25 Apr 2012 16:15:00 -0400

The Anatomy of Blekko's Search Database:

Blekko’s CTO describes the requirements and design of the custom nosql database that powers its search engine. I’ll admit this is a tad bit over my head, but WOW. The author presents very complicated computer science topics in fairly mild language (helps if you’ve spent time with Lisp or map/reduce). Can’t wait for part two.

Tumblr Firehose - The Gory Details

Wed, 25 Apr 2012 10:38:35 -0400

Tumblr Firehose - The Gory Details:

derekg:

mobocracy:

Back in December I started putting some thought into the tumblr firehose. While the initial launch was covered here, and the business stuff surrounding it was covered by places like techcrunch and AllThingsD, not much has been said about the technical details.