Oracle DataFox consolidates AWS databases on Autonomous Database
Oracle DataFox replaces specialty databases and reduces complexity with Autonomous Database when moving the service to OCI from AWS.
“After moving from AWS to OCI we greatly simplified development by adopting managed services and reducing the number of third-party services we have to manage and support. In particular, by moving from multiple specialty databases to a single converged database, our overall administrative workload is reduced. I feel much more comfortable being responsible for only one database.”
Business challenges
Oracle DataFox Data Management provides high-quality B2B data and signals, which helps businesses grow by improving their sales, account-based marketing, and supplier intelligence. This data is kept current by pulling from multiple different public, third-party, and government sources, before enriching it with machine learning, natural language processing, and “human-in-the-loop” techniques.
DataFox is built on a microservices architecture based on Node.js and Python. An earlier version used four databases: MongoDB, PostgreSQL, Redshift, and DynamoDB running in AWS. Aside from the complexity of needing to operate four different specialty databases, a change in license terms with MongoDB meant that it was not possible to upgrade to a newer release. This created obvious security risks as well. A new approach was needed.
Why Oracle chose Autonomous Database
Moving to Oracle Cloud Infrastructure (OCI) enabled DataFox developers to use managed services for some use cases rather than building all those capabilities themselves. For example, OCI Monitoring replaced third-party monitoring services. And by using Oracle Identity Cloud Service, the developers were able to replace custom security code that they had written.
When it came to replacing MongoDB and the three accompanying databases, developers were able to replace them all with Autonomous Database for analytics and warehousing, reducing the overall administrative workload.
Results
Today DataFox developers deploy a wide range of OCI services to complete their solution, including compute, network, OCI Monitoring, Oracle Identity Cloud Service, Oracle Container Engine for Kubernetes, and Oracle Analytics Cloud. They also use both dedicated and shared instances of Oracle Autonomous Database.
The main customer-facing application is now deployed on a single Oracle Autonomous Database instance on dedicated infrastructure, provisioned with 64 OCPUs and using 50 TB of storage. This means production workloads are completely isolated from other tenants and gives complete control over the timing of any upgrades and patching. In addition to the production environment, development, sandbox, and staging environments also use Autonomous Database Dedicated. This enables full testing of any updates or patches before rolling out customer-facing production systems at off-peak times, and with no downtime. In AWS this application used a combination of MongoDB, DynamoDB, and PostgreSQL.
There is also one other instance of Autonomous Database on shared infrastructure, currently using 16 OCPUs and 8 TB of storage, and originally deployed on AWS Redshift. This is used internally for analytics and data science. Running this variable and sometimes heavy workload on a separate system ensures no impact to customers on the production system.
Standardizing on one kind of database rather than four delivered savings in time and effort. Instead of manually backing up, updating, and patching four different databases, Autonomous Database automates this completely. Even performance tuning and troubleshooting is simpler with one underlying database rather than four.
Autonomous Database can do more processing at the server, rather than in the client, as was the case on the AWS implementation. This reduces data movement and can increase performance in some use cases, particularly with larger datasets. For example, account scoring migrated from MongoDB and PostgreSQL to Autonomous Database and now runs 10X faster. These performance improvements have made the system more stable, reliable, and predictable.
DataFox migrated from AWS to OCI and took advantage of managed services that simplify ongoing work. In particular, developers were able to replace multiple specialty databases on AWS with a single, converged database, and reduce the number of third-party services to manage and support.
Deploying the internal analytics on Autonomous Database Shared enabled isolation of this variable workload from the production system. Deploying the customer-facing workload on Autonomous Database Dedicated meant that DataFox developers could rely on autonomous management, but still retain the control needed to meet the needs of a 24/7 application.