Self-Designing Approach for Managing Large Datasets
Key-value stores are used to store and access data in large distributed databases with applications across social media, cyber security, transaction processing, and machine learning algorithms, forming the critical data storage component in data-driven applications. Existing database designs are sub-optimal, requiring more hardware resources and processing time. CrimsonDB, a startup emerging from Stratos Idreos’ Data Systems Laboratory, is developing software tools that allow existing applications to perform more than 10 times faster, compared to existing systems, with performance improvements increasing over these existing systems as datasets increase in size. This technology requires fewer hardware resources (less memory) to produce the same or better results, and is adaptable to new hardware and workloads, automatically tuning the system without human intervention. CrimsonDB aims to provide a dramatically faster and easier-to-use data system for optimal performance and costs in the cloud.