Monday, September 21, 2009

Building a Scalable Website

SecondTeacher is a website which will have very definate peaks in loads that are easy to predict. This is a great advantage in building the website but it also poses some challenges.
First we want to run the site efficently and on a budget that doesn't comprimise quality and user experience. We know the site will be busiest between 3pm and 9pm when the majority of students do their homework. In planning for peak load of say 20,000 concurrent users we can evaluate how much each machine can comfortably manage, imagine it's 1,000. We will then need 20 machines load balanced to operate the site. Correct ....?
Well no, we can do alot better than that and we also have a few concerns in relation to bandwidth usage. 20,000 people watching a 500kps video concurrently will need a hell of a lot of bandwidth if our 1,000 users per machine are all watching video then we would be pushing 500mps. Most hosters will only give you a 100mps burst capable pipe at a reasonable price so suddenly at best our machine can only handle 200 users meaning we now need 100 machines to cope....this is getting expensive.
Well our solution was up in the clouds.
First off we pushed all images and video to Amazon cloudfront which allows us to put all these objects on the edge locations close to the customer and also they are served from there meaning they do now clog up our bandwidth. This means we are now only in effect serving html and the silverlight controls now bandwidth is no longer a limiting factor. Now we're back to 1,000 per machine if anyone is keeping count.
Next we implemented our servers in the Amazon EC2 cloud meaning we could using there elastic computing model expand and contract our resources as we wanted to and only pay for the resources we use. This means that at 4pm on a school day we can run 20 machines but at 2am we only need run 1 machine to keep the service available for the desperate or slow students doing they're homework at this time. Our peak hours are only 30 hours per week the rest of the time 1 machine is enough. This reduces our resource costs to an average of just over 3 machines compared to the 20 we'd need to run if we went the traditional route.
But this is not all about cost, there are big advantages too in using the the Amazon cloud 1) Amazon engineers working 24/7 to keep the cloud up 2) Multiple Redundant storage and backup built in 3) No more concerns about hardware issues, if a machine goes down just start a new one. 4) Fully scalable e.g. if all our Christmases come at once and 100,000 try to access SecondTeacher in the morning instead of a technical disaster I can just fire up another 100 instances and the user experience is the same as ever.
I'm banking on the cloud to become the way of the future and I'm not alone.
Check out Amazon webservices for yourself Amazon Web Sevices

7 comments:

  1. Hello,

    first of all, GREAT JOB...

    I would like to build an infrastructure similar to yours using ASP.Net, SimpleDB and i would like to make it scalable like yours, but i have a question. I have read a lot about the latency of SimpleDB, my question is: If a user registers in your web site how you solve the latency problem? I understood that you store the sessions in Simple DB, but if some reasons there is some latency how you manage it?

    Thanks

    --silvano

    October 16, 2009 7:46 AM

    ReplyDelete
  2. Hi Silvano
    Thanks for the nice comments. Nearly all the interaction on SimpleDB is from our Web servers which are on EC2 so this has no real latency issues as they exist on the same network. We also cache all the main SimpleDB request on the server so SimpleDB is not even hit in these scenarios.
    Our Session state is held on SimpleDB too but again the interaction is between webserver and SimpleDB so again no problem.
    We use webservices to supply data to Silverlight controls on the client but we have no more latency issues then if we were running the DB server in a traditional way and of course we using caching extensively here too.
    On a side note I have used simpleDB for awhile now and I find it rock solid reliable and latency is not really an issue for me yet anyway.
    Thanks
    Robert

    ReplyDelete
  3. I have been evaluating EC2 for a while now. The question I have is in regards to run away costs. We build and maintain a content management system for websites. We have a few hundred customers. Some with very high traffic websites, others with small websites with minimal visitors. Quite a few with Videos that are streamed. What I'm concerned with is cost. Porting our system over only to find out that because of our usage, we are worse than we were before cost wise. In a month, if you were to consider your high usage periods when you have 20 servers provisioned, how much does that run you cost wise? Or how does that compare to the low usage periods?

    My problem is I just don't know enough about EC2. Even though I can read all of the white papers and pricing charts, I just don't know how to apply that to our situation and resource needs.

    When I saw your report, I got excited because now I've found a real person that has experience going through what we will need to go through and can help me understand the costs and how you performed your evaluation of need. I appreciate any help or insight you can provide.

    ReplyDelete
  4. Hi Robert,

    thank you very much for your explanation! You have been really very nice.

    Any idea of expanding in Italy? My father is a Math Professor at University of Engineering?

    Thanks Again

    --silvano

    ReplyDelete
  5. Hi Bilal
    First thing I'd say is EC2 is not a cheaper option. I have used The Planet (http://www.theplanet.com) to hire dedicated boxes for another project and they work brilliantly well.We have been running 4 boxes in the Planet for the last 3 years without a single issue. They are also very cheap so I would never complain about the service we get.
    Saying all that we are in the process of moving them to EC2 why? because of two main reasons 1) Peace of Mind. If a box on the planet goes pop we have a stessful time to rebuild and test etc to get it back up. In EC2 if a box goes pop we just launch it again. 2) Scalability, If we need to expand our resouces in the Planet it's a big undertaking that takes management and time, in EC2 we just launch up another instance and attached to the Load Balancer (which is great in AWS). Cost wise it's probably cheaper to go the Planet to tell you the truth.
    But we also push all our assets to CloudFront in AWS so I find it's much more reliable and quicker for our customers too. Why don't you try parts of AWS first like ther CloudFront and S3 to see how you get on.
    Hope this helps?

    ReplyDelete
  6. Hi Silvano
    Thanks for your comments. Italy is a way off I think my Italian is not good ;-)
    Ciao

    ReplyDelete
  7. Online home work help is a guide about working at home using the internet. Here you can find information, advice and help on online home work, work from home.


    http://www.homeworkhelponline.org/

    Akshay

    ReplyDelete