Improvements on the production environment

We recently made a few changes to our production infrastucture, detailed below.

Thank you AWS, Hello DigitalOcean

Since last saturday, we moved from Amazon AWS to DigitalOcean.

Previous environment:
Django over wsgi served via Apache, with a Postgres database, all running on a single micro AWS instance.

With this setup, FS was able to serve 2 requests/second.

New environment:
Django over uwsgi, served via nginx, with a Postgres database, all running on a single $10 “popular plan” machine (1core, 1Gb memory)

With this setup, FS was able to serve 5 requests/second.

Of course, the goal was not to benchmark DO x AWS.
But I do have the felling that DO would win on a “more fair” fight.

Also, the DO instance boots ridiculously fast – in my experience, AWS instances have much longer boot times.
Overall I’m very happy with the switch, but we need to see how it will behave in the long term.
I’ll keep the infrastucture on AWS for a while, just in case.

New URL

We also changed the base URL: freedomsponsors.org instead of www.freedomsponsors.org.
That’s shorter and nicer.

Web server setup

The nginx+uwsgi setup was very straightforward thanks to this tutorial by Daniele Procida.
Really, it’s one the best tutorials I’ve seen in my life.
Thanks Daniele. Keep up the good work!

What I liked most about uwsgi is that I don’t need to restart nginx to deploy a new version.
I just need to touch the uwsgi config file (touch ~/frespo_uwsgi.conf) and uwsgi will respawn the workers – which happens very fast (less then 1 second).

Caching

A friend told me I should try and configure caching to see how much performance improvement I would get.
Now FS runs with a memcached cache. Django talks with the cache using a standard unix (file) socket.
I did a benchmark running “ab -c 10 -n500 http://freedomsponsors.org/” on my machine, and got 18 requests/s as a result – but the CPU on the server was still below 10%, indicating that the limiting factor was the network.
So I ran the benchmark on the server… and got 616 requests/sec
Now that’s nice!
No need to worry if FS makes it to front page on Hackernews :-)

Emails

On the previous setup, email was delivered with a local postfix installation.
Now, I still use postfix, but I’m using Mailgun as a relay host.

Database backups

No change here. A postgres dump is saved to Dropbox (and syncd with my computer) every 12 hours.

Deploying

I also changed the way deploys are made. I don’t need to ssh into the machine for deployments anymore.
I’ll talk more about that in another post.

That’s it.
If you have any suggestions to improve our production environment, let us know in the comments below.

2 Comments

  1. hmm, I am getting a 502 bad gateway when i try to log in right now but I am not sure if it is related to this.

Leave a Reply

Your email address will not be published.

*

CAPTCHA Image

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

© 2017 FreedomSponsors

Theme by Anders NorenUp ↑

Follow

Get every new post on this blog delivered to your Inbox.

Join other followers: