Join Corey Prowse from Citigroup and discover how Citi architected a Market Data System (Scala, Java, TypeScript) storing billions of data points , scaling up to over a thousand concurrent connections within an Agile DevOps environment.
Challenge:
Manage billions of data points on a timeline of around 20 years
Support queries that return within milliseconds that are either single data point in time, a sequential block of time, or “walk” the timeline in a pattern
Provide real-time calculations and enrichment during queries
Deliver response times less than 100ms
Can scale up to high number of concurrent queries
Technical solution/architecture:
Reactive Streams technology that integrates with Couchbase NoSQL reactive technologies
JSON-specific optimizations