Select Page
FacebooktwitterlinkedinyoutubeFacebooktwitterlinkedinyoutube

Companies deploy Amazon Redshift for large-scale MPP (Massively Parallel Processing) data analytics. These systems apply parallel compute resources to answer queries quickly. However, there remain application security and performance challenges.

Heimdall Data offers an Amazon Redshift proxy to solve these challenges without code changes. As part of the Amazon Redshift Service Program, we are the ONLY Redshift-approved proxy. Deploy us with confidence.

 

 

 

  Use Case Heimdall Proxy Feature Customer Benefit
1 High connection counts Advanced Connection Pooling Improved Redshift scale and reduced memory and CPU utilization
2

Manual configuration of user roles

Regulatory non-compliance of data

Active Directory / LDAP policy enforcement SoD, FINRA, HIPPA, and SOC 2 compliance
3 Slow response times of repetitive queries Client-side SQL results caching SQL offload for improved scale and enhanced user experience
4 Slow queries consuming Redshift resources Workload Management Ensure SLA guarantees
5 Scaling Multi-tenant, SaaS platforms Multi-tenant Query Routing Improve Redshift scale with no code changes
6 Singleton INSERT overhead Transparent DML batching Increase DML throughput 20x

 Table I: Heimdall Proxy Use Cases for Amazon Redshift 

 

Connection Pooling & Multiplexing

 

Connection pooling and multiplexing save you Redshift costs by improving processing power. It is important to maintain the correct ratio between application and database resources. Each connection to Redshift takes valuable resources that can be better utilized for active queries.  Therefore, reducing the total number of connections to Redshift is desirable. 

Two techniques reduce connection overhead:  

  • Multi-user connection pooling: When a client establishes a connection, an existing connection is reused. When the client-side connection finally closes, the backend Redshift connection is kept open to prevent connection re-establishment overhead.
  • Connection multiplexing is an extension of pooling. As client connections are often idle, multiplexing establishes only “active” client connections to Redshift. The net result is 1) Lower memory utilization, 2) More active queries processed, and 3) Reduced Redshift costs.

Check out our AWS Connection Pooling blog.

 

Authentication, Authorization, and Auditing  

User roles are manually configured by the data teams. This resource is intensive to manage the groups/roles and is also an SoD compliance issue. The Heimdall Proxy provides:

1) Centralized Authentication/Authorization of users against Active Directory / LDAP

2) One-click report of users accessing sensitive data

3) Visibility and control of data access with approval chains

Check out our AWS Authentication/Authorization Policy Enforcement blog

 

Automated SQL Caching for Amazon ElastiCache 

The Heimdall Proxy provides SQL results caching logic for Amazon ElastiCache. We cache at the application tier, removing data latency. Our distributed architecture makes caching scalable while bringing the data one-hop closer to the user. Best of all, zero code changes are required; deploy in minutes.

Check out our AWS auto-caching blog.

 

Transparent DML Batching 

 Improve database write performance by batching singleton INSERTs against a table under a single transaction. If your application has inefficiencies of single INSERTs, batching will: 

  • Improved application response times due to fewer commits
  • Improved DML scale 

Ideal use case: Insert/update/delete a large amount of data at once on a thread. We can process it all much faster than if individual queries outside of a transaction were completed. 

Not-so-ideal use case: If there are concurrent writes and reads against the same table, on the same thread, everything will just block until the DML operation is completed anyway. 

If an exception occurs in the transaction, the proxy determines which query resulted in the transaction. It will remove the query from the list, report the exception in our logs, and reprocess the batch without the query.

 

Fast materialized views for results for Amazon Redshift 

Fast materialized views are very important in analytics environments.  When reports are generated, a subset of data is pulled from the back-end data store, then various operations are performed on that data. Heimdall provides the following functionality: 

  • Queries against a materialized view can be routed to an alternate database, typically Postgres, which acts on behalf of Amazon Redshift. Postgres answers queries offloading Amazon Redshift.
  • Heimdall triggers a refresh of the view automatically. Heimdall is aware of updated views from Amazon Redshift and when data was loaded that may impact the view. The net result is faster reports and a lighter load on Redshift, allowing the processing of other queries to be faster and more scalable.

The Heimdall architecture was designed for ease of deployment without modifying the application or database. Configuration changes are updated at runtime without restarting the application. The net result is a platform that can be updated with 100% application uptime.

Heimdall Data is an AWS ISV partner providing SQL security and performance in a database proxy. Available in the AWS Marketplace, we are integrated with Amazon Redshift, Aurora, RDS, ElastiCache and CloudWatch.

 For more information about how Heimdall Data, contact us at info@heimdalldata.com.

FacebooktwitterlinkedinyoutubeFacebooktwitterlinkedinyoutube