Systems such as Amazon Aurora provide dynamic scaling and performance. Another blog covers horizontal (more servers) and vertical (larger servers) scaling and the trade-offs. Database scaling can be costly, especially for commercially licensed databases. Others have used Amazon ElastiCache to improve responsiveness. However, developers are still challenged to know what to cache, what to invalidate and ensure data is up to date. This requires manual application code changes. We can now automate caching and invalidation for Amazon ElastiCache with Heimdall Data.
Heimdall Data is an intelligent query routing and caching data access layer that is installed in a distributed way on each EC2 instance.
Heimdall Data Software Packaging:
- Database proxy for RDS: Aurora, MySQL, SQL Server 2008+, PostgreSQL
- JDBC Driver: Oracle and any other JDBC complaint database.
For deployment, the only application-level change is to modify the host/port or JDBC URL to route through Heimdall. Figure 1 is a sample architecture diagram for a MySQL proxy configuration.
In proxy mode, there are two types of deployment:
- Distributed mode: A proxy residing on each EC2 application instance, for optimal performance (shown above)
- Centralized mode: One EC2 instance proxy servicing many application servers
The proxy provides two levels caching: 1) locally on the application server and 2) on ElastiCache (akin to an L1/L2 cache). As SQL is sent from the application to the database, the proxy responds from the cache, and routes requests to different servers (for use in load-balancing and read/write split). All this functionality is provided by Heimdall Data requiring zero code changes.
This article covers how the Heimdall system with Amazon ElastiCache and RDS can be deployed with an existing application. You can find a demo instance here.
Users testing with their own infrastructure require:
- Amazon RDS database (RDS setup instructions)
- Amazon ElastiCache Redise (Elasticache setup instructions)
- Heimdall Data instance. Various install methods are available at http://www.heimdalldata.com/do
wnload-self, including via the AWS Marketplace.
Script Installation of the Heimdall System
For users installing Heimdall via the one-line installation process, it will download and install the Heimdall Central Manager and proxy. The default user id is “admin”. If an Amazon instance is successfully detected, the password will be the instance id, otherwise the default password is “heimdall”.
The example uses an Aurora database supporting a WordPress application. Users can explore a live site without setting up their own infrastructure here. Role-based security will prevent modifications to the site but users can explore all features provided with the sample application setup. Testing in your own environment will require configuring an instance as detailed above.
Once the Heimdall Central Manager (HCM) is running on an instance, access with the server URL and port 8087. For an already configured server, the default tab is the Status tab which displays current server and system status. For a new installation, users are directed to the Wizard tab.
Heimdall has a wizard designed for Amazon. Select AWS Detect, otherwise perform Manual Configuration. The goal is to connect the database system and caching infrastructure.
If you see a screen requesting AWS IAM credentials, credentials can be added in this window or the IAM credential added to the instance as an IAM instance role through the AWS interface. Then, select AWS Detect again.
Heimdall Data safely, automates caching for Amazon Elasticache. Configuration is simple requiring zero disruption to the application or Amazon RDS. Users will experience up to 5x improvement in performance and scale. Heimdall is available as a free trial on Amazon Marketplace or downloaded at the Heimdall Data website.