Thursday, December 1, 2011

Optmization Complete

I've optimized Linkpad, and hopefully there won't be any more outages. We'll keep an eye on it. You may notice a slight increase in the speed of loading your page. If you have any difficulties, please email support and I'll do my best to help fix any problems.


Disclaimer: technical discussion ahead.

The optimization efforts did two things:
  1. I reduced the overall number of calls to the GAE datastore. In the code, I noticed several places where I was making redundant data calls. I consolidated them into a single call for each page view. (This should drastically reduce the load on the datastore).
  2. I enabled memcache. So, if you view your page without editing it, the results will be cached (and not fetched from the datastore). As soon as you edit your page, the cache is cleared. No stale data here! Introducing memcahce further reduces the load on the datastore, as it caches your page.
I will keep an eye on my datastore load over the next few days. If the load is low enough, I will turn off the premium service, and Linkpad can revert to being a cost-free operation.

EDIT - further optimization:

I implemented further optimizations which solved the root of the issue. Turns out I was blowing the quota for small datastore operations. These operations were being generated by calls to count() all over the code. I was able to work around all of these except those on the /stats page. I had to remove the stats about how many users and pages Linkpad has, as these stats were causing the application to go over it's quota.

Linkpad is now back to being a cost-free operation.