In December of last year, just as the MobileCSI brand was being reborn as SceneDoc – we began a similar project on the engineering side to re-imagine the SceneDoc Cloud from the ground up.
The SceneDoc Cloud was originally written on a PHP/MySQL/memcached stack. This setup fit the bill very well for our design and implementation constraints at the time. The speed of development was very fast, the ecosystem around PHP is very mature and the choice of libraries is enormous. We needed to move fast, iterate quickly and deploy often in order to get a prototype out the door and begin the demo/trial/sales process. We used a simple MVC framework which enabled us to create new parts of the system very quickly.
The further down the road we went with this proof-of-concept system, the more quirks and nuances we found which limited our ability to “scale”. Not “scale” as in able to adapt to heavy load, but scale from a business perspective. We had a laundry list of must-haves that were largely customer-driven and we were unable to feasibly make them happen with our POC system.
Our laundry list included:
- Database agnostic (for at least MySQL, MSSQL and Oracle)
- Secure out of the box
- Good separation of concerns (e.g. the database layer shouldn’t need to know about web sessions)
- “Pluggable” i.e. heavy use of Interfaces to enable us to swap parts of the system in and out easily
- High performance while remaining stable
After much debate, we finally settled on the Java Enterprise Edition (JEE) platform as the base for our product.
The new system was completely rewritten from scratch with security in mind from the get-go. From day one we took measures to prevent Cross-Site Request Forgery (CSRF), Cross-Site Scripting (XSS), Session Fixation, and Clickjacking via XSS and frames. Additionally we added configurations for HTTP Strict Transport Security (HSTS) to our application to match our SSL configuration.
With all of these attack vectors already mitigated, it allowed us more mindshare to focus on our actual business logic rather than the minutiae of hardening our systems.
When we moved our system from PHP to Java, our entire codebase became a lot more loosely coupled. In an ideal world loose coupling means that each part of the system does its job and doesn’t worry about anything else. This allows engineers to focus on one thing and make changes to the internal workings of that thing without affecting the entire system.
Previously the system was architected similarly to a house of cards – if you remove (or break) one part of the system, it could have many unforeseen consequences. The new system is built more like a brick wall – if one brick is removed (or broken), the wall will still stand and you can have an engineer come back and fix the broken brick.
In network infrastructure lingo, scalability is the ability to adapt to high load. We could always do this with our old PHP system – responding to load is a matter of simply adding more front-end servers to our load balancing pool, and the same is true of our new system.
The new system however, impressed us with its ability to handle high load without breaking much of a sweat. Consider the following graph:
Response time vs Active Users – graph generated by http://xkcdgraphs.com/
In our tests, we discovered that the PHP system had very quick response times at a low utilization, but the response times increased exponentially when adding more load. Performing the same test on the new system yielded a much more stable system overall as the response times – while higher than PHP at low usage – remained steadier and increased in a much more linear fashion with load.
This means we can serve more users with less infrastructure.
What does the future hold for SceneDoc? We have many exciting new features on the roadmap that all take advantage of our new system’s architecture:
- Full 2-way synchronization of data
- Multiple API endpoints and formats for consumption by other systems
- Team-based collaborative data collection
- Much much more…
Keep an eye on the SceneDoc website and Twitter (@scenedocinc) for more information of these developments as they happen.
The post Security, Stability and Scalability – why we rewrote the SceneDoc Cloud from scratch appeared first on SceneDoc.