When I get my inbox to zero, I’m like
via Shirley
Nick Fishman
I'm a software engineer and entrepreneur. I like to solve high-impact problems with technology. I'm the CTO and co-founder of ampcloud.
I'm a software engineer and entrepreneur. I like to solve high-impact problems with technology. I'm the CTO and co-founder of ampcloud.
I propose a revised version of the famous proverb: “The way to a man’s heart is through his startup.”
Coding in the cloud (almost)
Plink is a very original idea. It’s an online, collaborative, multiplayer music making toy made byDinahmoe. It uses Node.js and WebSockets to create an multi-user “chatroom” but instead of entering text to have a chat, the interface generates music!
Pulsating circles are generated by moving the mouse over a <canvas> element. Clicking and holding the mouse generates a musical tone. The colour of the circle determines the type of audio you play such as high or low notes, and is created using Google’s Web Audio JavaScript API. What a great combo of web technologies, and great fun too!
Check out a video of it in action: http://vimeo.com/26271666
There are some cool things going on here. On the client, they’re using the Web Audio API (available in recent versions of Chrome and Safari) to dynamically play sounds, and WebSockets to make the experience live and interactive. They’re using Node.js on the server.
It’s worth noting that the sounds all come from a pentatonic scale. This is why the music miraculously doesn’t sound cluttered or discordant, even when lots of people are playing. You’ve probably experienced something similar if you’ve ever tried playing just the black keys on a piano: no matter what order you play them, you just can’t go wrong.
Each instrument is sampled across 16 tones. Try some of them to hear what I mean:
On a tech note, the client code isn’t the cleanest thing in the world. It would be easier to read (and maintain) if it used Socket.IO instead of raw WebSockets, and if it used jQuery or some other JavaScript library to manipulate the DOM.
Still, a very innovative use of the Web Audio API and quite fun to play with.
I’m currently building a startup (ampcloud) with Node.js, MongoDB, Mongoose, and a handful of other tools. After spending quite a few years in the Django world, it’s been fun doing a mental context switch into the land of JavaScript, callbacks, and closures. Occasionally I’ve run into some gotchas, and this particular one is a great example.
Let’s say you’re building a blog, and part of your database schema looks something like this:
var CommentSchema = new Schema({
title: {type: String},
body: {type: String},
createdAt: {type: Date}
});
var PostSchema = new Schema({
author: {type: String},
title: {type: String},
createdAt: {type: Date},
slug: {type: String},
comments: [CommentSchema]
});
module.exports.Post = mongoose.model('Post', PostSchema);
Every post is stored as a separate document in MongoDB, but all comments are embedded within it. This means that when you fetch a post, you’ll get all the comments back with it.
Now let’s say you want to display a list of the 20 most recent blog posts on your home page. Assuming you’re using Express, you would write a view like:
app.get('/', function(req, res) {
Post
.find()
.asc('createdAt')
.limit(20)
.run(function(err, posts) {
if (err) {
res.render('error', {status: 500});
} else {
res.render('allposts', {posts: posts});
}
});
});
You’d also want to add an index to allow efficient querying by date created:
PostSchema.index({createdAt: 1});
Your blog will probably work well at first, but you’ll run into problems as soon as one of your amazing posts goes viral and gets thousands of comments. You’ll notice that your main page starts taking a lot longer to load. Even when you’re the only one browsing your blog, it just won’t feel as snappy anymore.
The culprit is the comments field. Because a Mongoose query requests all fields of a document by default, every site visitor will cause it to request and parse the entire list of comments. Every time. You don’t even need the list of comments to render the main page.
Let’s get rid of the comments field by adding the following line to the query chain:
.exclude('comments')
The final result:
app.get('/', function(req, res) {
Post
.find()
.asc('createdAt')
.limit(20)
.exclude('comments')
.run(function(err, posts) {
if (err) {
res.render('error', {status: 500});
} else {
res.render('allposts', {posts: posts});
}
});
});
You’ll find that this performs a lot better. The problem isn’t so much that MongoDB can’t return the data quickly enough. Rather, Node.js has to spend much of its time parsing extra JSON into JavaScript objects, which is both unnecessary and time-consuming.
Not surprisingly, I recently encountered this issue in production. I made the fix right at 3:00 GMT, and the load dropped dramatically.
When your models start accumulating lots of data, think about whether you can request a subset of fields when making queries. See the Mongoose query documentation for details.
Caveat: Keep in mind that you won’t gain much by excluding fields that store primitive types like Strings, Numbers, or Dates. Even worse, your code will probably get harder to read and maintain. Only make such optimizations when you have to.
The above schema suffers from a fundamental flaw: it doesn’t scale well. If a blog post gets thousands of comments, you’ll probably want to paginate the comments and only show several hundred at a time. But with this schema, you can’t ask MongoDB for a subset of comments. You can only get all or nothing.
To make this production ready, you’d probably want to separate Comment and Post into separate Mongoose models, instead of nesting Comments within Posts as embedded documents. Each Comment would be a separate MongoDB document, you’d store the Post id within the Comment, and you could efficiently query for random subsets of comments on a particular blog post.
Greetings, fellow tumblrs and other readers.
I’ve created this space as a way to share interesting, sometimes relevant, occasionally whimsical, and hopefully useful thoughts about technology, entrepreneurship, people, music, and other important life matters.
Ideally, this would be part of a personal website that looks flashy and represents me perfectly. There’s a Russian proverb that my parents bring up at times like these:
“Лучшее — враг хорошего.”
“The best is the enemy of the good.”
—Voltaire
With that in mind, this will do for now.
