Scaling Livemocha: Part 2

If you are interested in some of the initial steps we took towards scaling, here is Part 1

Improving Latency

Over the past year we have done lots of work in optimizing the latency for our users. There are two parts to the total latency - Server and Client Latency. While you should improve both, its important to keep in mind that 80+% of the time the user sees is on the client side after they have got the initial html back from your server. For e.g. most of our server responses will be in less than a second so if the page takes 5s to fully display over 80% is spent on client.

Livemocha also has a harder challenge in that our users are from 200 countries so a page that can be rendered quickly in US and UK maybe unacceptable in China.

The first step we took was to add hooks into our product pages to measure our response times and times to load the pages with all resources. This gave us baselines to compare with.

Optimizing

A good initial step to optimizing your site is to use the recommendations from YSlow and Google Page Speed Tool. Running these with your site will tell you what to do specifically.

• Good practices that we were already following were things like: Gzip resources, setting the right expires, etags, minimizing # of requests, sizes of files, minifying etc..
• At some point we had to start doing the harder things like combining resources, image sprites etc.

A lot of our efforts went into setting up the infrastructure to capture latency benchmarks and report on it. This allowed us to fix country specific issues as well. The latencies for our top pages are monitored constantly to identify issues or track improvements.

While this is still a work in progress for us, we have already made significant improvements in our site latency and the time it takes to render pages on our site.

In a nutshell:

• Setup measurement frameworks. Don't guess. You may have to home grow some parts of this based on your needs in addition to using standard tools like gomez.
• Use the recommendations of YSlow and Google pagespeed tool. E.g. This home page that has done all the optimizations that YSlow recommends: Only 1 JS, 1 CSS, image sprites etc. and the page loads quickly -even though its hosted in a different country.
• Some changes will have more bang for the buck do those first - e.g. gzip, expires headers. Doing image sprites and combining scripts are way harder. Do the easy things first.
• Adding CDN can make big difference -if you have a lot of static content. With AWS cloudfront, you can easily add this if you need to.

What's in a name?

Are there good names for your startup?

We picked the Livemocha name during one of our meetings in Tully's coffee shop ( which we worked out of initially). It represented the lively conversational ambience in a cafe. It tied in with our vision for Livemocha that our users should be able to walk into a cafe anywhere and be able to carry out a conversation.

Over time the name grew on us. In fact all our alpha users loved the name too - the Livemocha name stood out in all our surveys.

How do you select a name from your options? I believe the best names are those that are not too closely tied with the product. E.g. Google, Apple, Amazon versus say LanguageExchange - In fact it could be problematic if your brand name has the product in it and you are trying to get away from that. Imagine if Bezos had picked ebooks instead of amazon, they would have trouble positioning themselves as a general retailer much less be a company that provides cloud computing.

In some cases if there are a lot of names in space, then having a generic name could help you stand out. While this is risky since the consumer has to figure out what you do, it definitely won't be mistaken for another one. E.g When we considered a name , there were a ton of companies with names like LingoX, PlanetLingo etc. Picking something different helped us stand out. Ofcourse you don't want to go too far - During 2007, names of web 2.0 companies started getting crazy and I would definitely stay away from names that make it hard for someone to type even after knowing the name. Something simple like LetsTalkNow is simpler to remember than PLingrr.com


Of course you have to find domain names and unfortunately due to how lucrative squatting and ads from mispelled sites are - this is unlikely to diminish. As a result, you have a double whammy not only do you have to find a good name but also be lucky enough to find the domain name. You could try to make a bid for a name that you like but if you are strapped for cash, you wont be able to spend more than $1000 on getting a name.


If you can find two word names that should be good enough. There are a lot of tools online that generate synonyms etc. and can check if domain's exist right away etc.

Overtime you will get used to your name and like it - don't worry. If you are successful, you can always create a new name :)

Best practices after you pick a name

Register misspellings: One thing we did not do quickly enough and I would recommend doing is to pick mispellings and register them. You will be amazed how quickly they go once you create a company with a name. I think the domain registrars are running some automated program. E.g. the day after I registered livemocha.com - livemocha.net was taken. Some misspellings were taken by the time a press article was written about us. Ofcourse you don't need to go crazy and register 100's of them right way.

Register trademarks: This may be something you can wait but you should figure out how important it is for you and do it quickly then.

Livemocha Series B funding

We just announced our series B funding news. You can read more here - Press Release

I am very excited about the prospect of having August Capital and Howard Hartenbaum on our board. Howard has an excellent track record including investing in Skype, Photobucket, Bebo. He has a lot of experience with managing a global audience and will help us scale our business.

Looking forward to the next year as we continue to make progress in our goal of becoming the place to learn a foreign language. If you have not checked us out do take a look - It is the most natural way to learn a language, you learn with engaging online material and practice with real native speakers.

Starting something: idea evaluation to starting

Before starting Livemocha, I always thought the idea for starting something would come in an aha moment ( followed by a frenzy of activity building the site and launching to success).

The idea for Livemocha did not happen overnight. It took a lot of steps to get to that point. I was recently thinking about this process as one of my friends was asking me for my thoughts on his ideas for a new venture.

For someone starting out new - how do you get from a general idea that you want to start a company to your vision, then product and eventually market.

If you want to launch a product/company where do you start?

Brainstorming for Ideas

Are there are any areas you are passionate about or know a lot about - e.g. Say you have been teaching at Kaplan for GMAT preparation and think there is a better way to do this or you have been taking guitar classes for a long time and think there is a better way to do this online or through mobile. Maybe you care about solving the issue of kids getting better math education and want to build a site for that.

Other options are to consider solving problems you are facing or services you will find useful. This usually will be a consumer product. E.g. I find it hard to find nannies through craigslist - maybe there is an opportunity to build a service to do that. If you or someone in your team has been dealing with customer requests or sales in the enterprise area and have an idea of what products people are asking for - that could be an enterprise opportunity.

When you are considering ideas, write these down. You don't need to evaluate or dismiss them immediately. Consider the most promising idea and start to flush it out. When we were starting Livemocha, a promising idea would usually take weeks to flush out.

Evaluating your ideas: How do you identify the good idea

Essentially each step in this process of evaluating your idea is to remove the various risks associated with a startup and try to pick the one with the best shot. There are external risks (market, regulations etc.) and internal risks (team, execution, raising capital etc.).

Step1: The first step that I recommend is to do a search if someone else is doing the same thing. More often than not, you will find someone else has had a similar idea and is already out there. Usually if they have 6 months lead over you and that's the only thing they are doing it will be tough to catch up ( unless you can raise more capital or you have a unique twist on the idea). How crowded is this marketplace?

• If there is already someone doing this - estimate the amount of traffic they get and registered users etc. Essentially how big are they? You can use alexa, comscore, compete to guess how much traffic they have. Look at the management team, the product and see how they have executed.
• Any other market reports? For e.g. when we started Livemocha we found lots of market research estimating the size of the market, distribution of learners etc.

• If someone else has already done the same and you don't have a significant differentiation, then you should consider something else instead of taking them head on. If they are already a well funded company with a few million users, it will be hard to take them head on. You have to figure out how you are going to approach this space.

Ofcourse there are always exceptions - Raj from Mayfield said there were tens of companies doing photo sites when he started snapfish. They did very well but most of the others died. When we started Livemocha, we did a search and found a few companies doing online language exchange sites. The good news was that they were not doing exactly what we wanted to do and we thought we could differentiate from them.

Step2: If you feel you have a reasonable shot at building this service/product - evaluate the business risks. How big do you think the market is? What will your business model be? ( Its ok not to have this figured out but atleast hopefully you are building in a market thats growing and has revenue potential). I have a post about this earlier - this is an important step, do not build a website thinking you will be able to sustain it on advertising. Also think of people's propensity to pay - while 3d photo albums might be cool, its an unproven model whether people will pay for it.

If your idea has done reasonably on these first 2, evaluate the product risks. A good example is building a new search engine thats significantly better than google in say image search. The opportunity is definitely big but the product/technology risk is very high. For many websites the product risk is not high.


Next Steps:

At this point, if you feel like your idea is worth pursuing, you should start flushing out the next level of detail - what does the product look like, how long will it take you to build it. What team will you need. How much money do you need to launch it? Then you would create a business model based on your assumptions. How much would you charge, how many people do you think will buy. ( create a hypothetical model based on traffic and paying assumptions). How are you going to acquire your initial customers - is it adwords or PR (estimate how much it will cost etc.). Getting distribution is a hard challenge.

Differentiation or Barrier to Entry:

Assume people will copy your idea if there are not already 20 people working on something similar. Consider what barrier to entry you can create with your product? This way when someone does copy your idea ( and they will soon) - how will you sustain. These barriers can be network, content, operational, low cost of customer acquisition etc. E.g. Its not very hard for Microsoft or Google to build a site like ebays but its going to take a long time to get over the network effect of - merchants/buyers.

If its only going to cost you $50k to launch it and there is no barrier to entry then you have a challenge as others will get in and its hard to compete. E.g Lets say you are building a site where you are categorizing youtube videos yourself. Assume the site costs 50k to launch and 10k monthly to continue categorizing. What is your barrier to entry if someone else does the same thing? One idea would be to have your community users also start doing it, that is hard to replicate once you have a 100,000 users doing this.


Go through the steps in detail. Its better to have wasted 2 weeks and throwing away an idea than wasting a few years chasing down the wrong path. We threw away a lot of ideas before Livemocha passed our litmus tests. It took us a good 2 months from the initial concept to finally making the decision and in that time we had a lot of stuff flushed out.

One thing I have learned over the past 3 years, is that no matter what you choose to do, building a successful company is hard. So you might as well pick something you care about.

Top metrics you should care about for your website

Over the past 3 years at Livemocha we have been very diligent about tracking metrics on the site. Right from day 1, when we launched our site - we had both in house reports and google analytics, to help guide our decisions. While this data has been extremely useful in making decisions - over time the volume of data has exploded. On a weekly basis we review a lot of metrics in detail with the exec team.

However with the overload of data its easy to lose sight of the really important metrics. The ones from which everything else should flow top down - otherwise you risk losing the forest for the trees.

Lets consider a hypothetical online site that is a social networking/dating site.

The top metrics for the site to consider are:

a. Acquisition: Viral Coefficient or Conversion rate

b. Engagement: Key Goals Conversion rate - e.g. % of Users getting a blind date accepted.

c. Retention: Returning users from month to month

d. Monetization: Revenue/Visitor

Acquisition

Viral Coefficient: If the primary driver of user acquisition for your site is word of mouth or people referring others after using your product - then you should measure that carefully. Most folks think of distribution as an afterthought to building the site. It is a hard problem to get traffic to your new site (more on this later). For the dating site above, we would measure how many other users each registered user brings in. Andrew Chen has a great post on this: Viral Coefficient explained. Track this carefully from month to month. Ratios above 1 are really exceptional and it means your site is growing rapidly.

Conversion Rate: If your primary driver is SEM, you should measure your conversion rates carefully to understand how much you spend to get a registered user which then ultimately leads to how much you spend per paying customer.

Engagement

A new registered user may do a lot of different activities on your site, its easy to get side tracked with all those metrics - e.g. in above site, you may start tracking the number of pictures they upload etc. However first you should track conversion to the #1 activity that correlates with user activity. E.g. if its sending a message to another user that highly correlates with dating activity and the first step- then track that carefully - Do 80% of your new users convert to that goal. For a site like redfin, the key conversion rate maybe scheduling a tour with an agent.

Retention

Depending on the type of site you have built, you will have an expected lifetime of a user - e.g. Dating sites usually have a lifetime expectation of 3 months by which time you have found a date. A real estate site like redfin might expect lifetime to be 6 months after which a customer has found a home and has no need to use it. Facebook might expect lifetime to be years. Essentially track how many users who registered in month 1 come back the next month. That gives you a retention rate. Coupled with your acquisition rate, it will dictate whether you are growing, staying flat or dropping. Look at just monthly visitors does not give you any idea. I have heard lots of folks only measure monthly uniques. While that's ok to give to someone outside the company you cannot take any action on that number - is it new users who are 90% of that number , is it repeat users etc.. Each dictates a different action.

Monetization

Track the revenue per user on the site. E.g. if you have 500,000 new users registering every month and you make $250k, the revenue per user is $0.5

One last note - track all the above as cohort reports. Otherwise it is difficult to gain insights from averages. E.g. all 4 above metrics can be tracked for new users from Aug, then for Sept etc. to know progress. Create spreadsheets or graphs where you track the number of registered users per month and then track the metrics for each cohort separately. As you iterate, its easy to see it change for the appropriate cohort.

Building a new website: What should I choose - Rails (Ruby), PHP, Python...

You have a great idea for a website and have decided to make a go at it. What should you build it in?

When we started Livemocha , we spent some agonizing few days thinking about this. Realizing this was a critical decision - I was stressed out about picking the wrong platform.

We decided to go with php and a framework in php. We were comfortable with it, PHP community had solved/provided tools for many scaling issues and if we could use a framework that let us keep our code clean that would meet our goals.

For the most part over the past 3 years, it has met our goals and I definitely think we made a good decision.

However what I have learned over the years is that the most important factor you should consider ( assuming 2 frameworks meet the requirements similarly) is your comfort level. Go with the framework that you are most comfortable with. You have so many other risks when you start a startup (team, product, business etc...) - you don't want to take on more risks with your platform as well. For e.g. if you have been using php for the past 6 years and have scaled it well - there should be a really really strong reason to suddenly start using ASP.net to build your startup.

Take risks in stuff you are innovating...

Start thinking about traffic while you build your site not after

I was recently talking to a friend about new ideas for a website. As we talked through the idea and how useful the service would be - we hit a wall when we were trying to figure out how to get users to find out about the service. This is by far the hardest problem for new sites.

It is so easy to develop a site these days, how is someone going to find out about your site. No matter how cool your idea, spend some cycles upfront figuring out distribution. No it won't be viral. There are probably a handful of websites that are truly viral. Why would yours be?

As Howard Hartenbaum told me once 'To be truly viral either your product to be useful - requires someone to invite others ( e.g. skype, facebook) or by using the product you let others know - (e.g. acrobat reader, hotmail, picasa etc.)' - so if your site is not in one of these categories you have to figure out other ways to acquire traffic.

Some options for you are:

• Acquire traffic through SEO - A good example is scribd. Its not a truly viral product but I can bet it gets a lot of traffic by people searching for a certain book or phrase in google.
• Acquire traffic through SEM - You may have to spend money on adwords etc. to get traffic.
  

If you think either of the above 2 is going to be your path - then estimate the amount of traffic for keywords and amount it will cost you for keywords if you are going to use SEM

A third option is to rely on PR. This is usually a hit or miss strategy.

Once you have some traffic for the service to take off you need a few elements - product has to be good to generate good word of mouth ( Livemocha has incredible word of mouth with 1000's of blogs written about us). If it also turns out to be viral fantastic. Also the product needs to be sticky - only then will people return otherwise, you are going to keep having to acquire new customers.

In any case - spend a lot of time thinking about distribution right from day 1 - If you build they will come does not work.

Websites and A/B testing

Which is better for conversion:

OR

A sure way to fail is by designing without any data or testing. While everyone thinks they understand their customer’s needs, all too frequently folks within the company argue about what they think is the right feature. ‘Should we have this text section on our home page. Nobody reads text, to improve our conversion we need a cool flash section….’ And so on the debate continues..

In my previous company it was more important to get this right since we shipped a CD once every 6 months. Today in Livemocha: World's largest language learning site we can operate with a mindset that while we have a couple of hypotheses on what works, we really don’t know the right answer. Instead of arguing, let’s just test it out and the data will let us decide. We can often make a change, deploy to production and find out the answer in less than a couple of hours.

Its hard to internalize this mindset - we all have opinions and often think that we know the answer. Most of the time the answer comes from outside the building, from the users, from customers esp. when you have with millions of users from over 200 countries, there is no way a couple of people in Seattle are going to get everything right.

As a website engineer, you are like a scientist running experiments constantly trying to figure out answers. As a result its more important to focus on the process and make sure you have the tools to help you get data quickly and zero in on the answer than have answers right away. Here is the five step program to get there.

1. Adopt a Rapid Development methodology: Your product development process needs to be fluid to allow rapid iteration and measurement. This means that you are going to have to not only change things quickly but be comfortable that you are running some versions on production that are less than ideal sometimes. At Livemocha we follow an agile methodology. Our release cycles are extremely short and usually we try to get a minimum version that we think will be acceptable.
2. Testing Mindset: All the key members of your product development team need to subscribe to a 'I dont know lets test and find out' mindset. Its actually refreshing to think this way - Your website is a constant series of tests. You dont think of overhauling the entire page, you are always trying tests that give you the insights to make improvements. Instead of having endless debates, its easier to convince people if you tell them its a test.
3. Its ok to make mistakes. In fact, we made a key improvement in our conversion rate, when we made a mistake and removed one of the buttons on the enroll page. You should be ok with having some bad days and having less than ideal results. If you cannot afford for e.g. your conversion rate to drop more than a 1% then its harder to test than if you can.
4. Start with your key pages and big drop offs. Try simple different combinations. Monitor results closely. You need to spend some horsepower doing this.
5. Leverage third party tools: Use google analytics and website optimizer - these are free. There are no excuses for not doing A/B testing in a web company.

Tools we have found good results with:

• Google Analytics: You can use funnels and goals to figure out whats going on
  
• Google Website Optimizer: You can then identify pages on which you can for e.g. shuffle blocks around or display a different picture. The tool will run the experiment and tell you which is the winner to a specified goal.
  
• Use a tool that lets you see what users do on that page ( either through logs, analytics or tools like Clicktale)

Thats it. Happy testing....

Here is an example of how you can do this:

Lets say you are interested in increasing your visitor registration conversion. The first step would be to create a funnel in Google Analytics e.g. Home page - Get started page - Register page - Post Register goal page

This will tell you how many people move between these steps. Lets say you think a big drop off in people from home page getting to get started page. Maybe there are links to other places, maybe the text is not clear, maybe people are bouncing off that page.

At this point if the bounce rate is high - assuming you are acquiring the right set of folks, check the page for performance issues, load issues.

Then try to figure out different tests you can run on the home page - Lets say the home page like Livemocha has a main portion and 4 sub portions under it. Main, Left 1, Left 2, Right 1, Right 2. Now using Optimizer you can run a lot of tests e.g. Home page only has Main, or Home page has only Main, Left1, Right 1, or sections interchanged etc. This will tell you the winning combination w.r.t to the conversion goal.