Web and mobile applications typically contain a user-visible front end, a query generator, and a back end which does the data computation and serves the front end. In response to a user or API request, a web application interacts with the API or with persistent data stored in a database. The application must support different clients, such as browsers and mobile devices, and interact with other systems and apps by using APIs and events. The backend must be secure and scale on demand.
Build your web application as a set of microservices that can be independently tested, deployed, and owned by different application teams. Expose services as REST APIs and communicate with other microservices using a built-in event mesh for events & messages, or APIs created per microservice. Incorporate machine learning in intelligent apps with ML models through REST endpoints in the database.
Web and mobile apps need to be scalable to handle spikes in demand and operate under stress with low latency. They must be available 24/7, resilient enough to produce data when requested, and must not lose stored information.
Mobile Apps are built with front-end frameworks like React Native or SwiftUI, creating the user interface which interacts with a backend for all its data and enrichment services. The developer can focus on the workflow and the application logic, accessing the backend through well-defined APIs that are resilient, secure, and scale autonomously.
Security is of paramount importance for web apps exposed to the internet. Data encryption, TLS, DDoS protection, firewalls, and granular user and data access management are critical. Both database security and application security are important, and handled with passwords and keys accessed through Oracle Database Wallet and OCI Vault service.
A web or mobile application often needs to interface with other business systems and services that are outside the organization. Services that are part of a web application should enable interaction and collaboration via well-defined APIs. Use Oracle REST Data Services (ORDS) to use data APIs or build new ones. Use ready-made REST endpoints through OML services and OML4Py embedded Python execution to enable machine learning. Use available API Gateways on the Oracle Cloud as the single point of entry for all clients and route API requests to the appropriate service. Configure load balancer services and ingress controllers for secure API communication between microservices.
Maintaining a web or a mobile application infrastructure comes with the responsibility of deploying, upgrading, patching, scaling, and securing the setup. Use managed services such as Oracle Autonomous Database and a host of other managed services on the Oracle Cloud to maximize availability and scalability, and respond to the changing demands of web and mobile apps. Managed services ensure that the application is available 24/7/365 and protected when a failure occurs in the data center where the infrastructure is hosted. Use self-managing services only when a vendor- managed service is not available.
Keep middle-tier components of application stateless. If the application state is to be stored, use Oracle Autonomous Database to store application data and state for consistency, durability, and fast recoverability from the root of the application state. Keeping the state in one database is simpler and more efficient in overall application recovery.
Web and mobile applications use data in different formats, and they need to store, search, and process for the data in a data store. The data might be tabular (relational), unstructured, formatted as XML and JSON, spatial, or graph. Traditionally, this disparity meant using a relational database for relational data, a document store for unstructured data, and graph databases for hierarchical linked data. But the use of multiple databases leads to operational complexity and data inconsistency. To solve this problem, use the converged Oracle Autonomous Database to store multiple types of data, index them, and provide ways to search the data, and use for unified analytics across all data.
A web or mobile application can contain hundreds of services, owned by different application and business teams. Unified observability with open tool interfaces is important for gaining visibility into the behavior of these inherently distributed systems. Centralize the observability solution by using the metrics, logs, and tracing exported from all layers of the application and data tiers. These services monitor the entire stack, from front-end to backend helping find and fix issues in the application quickly, and become the platform for continuous performance tuning.
Web and mobile applications must be resilient, be able to recover from failures, and minimize downtime, and eliminate data loss. Redundnacy helps eliminate single points of failure: Use Kubernetes to manage the resiliency of database containers and app containers. The Oracle Database Kubernetes operator is designed for this, using CI/CD pipelines that include the data tier. In Kubernetes clusters, set up node pools with a minimum of three nodes with each node in a separate availability domain in a multi-availability domain region on OCI. In a single availability domain region on OCI, setup node pools in Kubernetes with each node in a separate fault domain. Use a single public load balancer and multiple private load balancers with multiple ingress controllers for redundancy in Kubernetes.
Setup the Autonomous Database for maximum availability using Autonomous Data Guard to minimize operational downtime and data loss.
Follow the principle of least privilege to ensure that users and service accounts have only the minimal privilege necessary to perform their tasks. Control who has access to the web application components by using Cloud Identity and Access Management (IAM) including the database. Use multifactor authentication in IAM to enforce strong authentication for administrators to restrict access to the application components and the database.
This converged data architecture pattern for a web or mobile application is composed of a front-end, and microservices with the back-end using the app and data services built in the Oracle Autonomous Database. Optionally, for services not in the database, microservices in containers are managed in a Kubernetes cluster.
The following diagram illustrates this reference architecture.
The architecture has the following components (Components can be fully managed services on Oracle Cloud Infrastructure or equivalent services in other deployment environments. For example, Oracle Autonomous Database is a managed service on OCI, and the Oracle Database runs in other environments).
This use case describes a sample mobile food delivery application called GrabDish that uses a Microservices architecture with the Autonomous Database, and containerized databases (Pluggable Database in a Container Database) for each of its services. It uses the built-in messaging platform Transactional Event Queues (TEQ) for messaging, simplifying the common microservices patterns for developers. GrabDish shows AI/ML services, and a variety of data types used, programming the app in multiple languages. For details see the Livelabs on GrabDish at https://bit.ly/bettermicroservices
Learn more about related resources.
Messaging solutions connect application components—including existing on-premises systems—to cloud solutions. Message payloads are events, which are application generated, user inputs, data changes, or device generated events. Messaging enables event and data transfer as part of a well-defined distributed processing pipeline, or for publication of messages and events to multiple independent downstream systems that process, enrich, and analyze data. Most modern apps built with microservices rely on an event-driven architecture. An Event Mesh allows for any event produced in a system to be securely consumed anywhere else in the distributed system where it is needed in near real-time, using one or multiple Event brokers. After an event is processed, the data (payload) is stored in a data lakehouse for analytics and AI/ML model training.
Build messaging solutions that are highly available, reliable, and flexible. Use Oracle's converged database platform, cloud services, and best practices to deliver messaging and even-driven solutions based on business needs. These recommendations help minimize development integration, deployment overhead, and long-term management burden.
Messaging solutions connect application components, including your existing on-premises systems, to cloud solutions. They enable data transfer either as part of a well-defined distributed and converged processing pipeline or publish messages and events to multiple independent downstream systems that evolve independently.
These solutions should also transparently accommodate unplanned spikes in message load by buffering data and dynamically adjusting process resources. In the past, it was challenging for enterprises to deploy and manage reliable messaging solutions that meet these objectives without undue complexity and expenses. However, this implementation can now be straightforward in the cloud with messaging services designed for scale and performance.
Use the following design principles to build your messaging applications or platform with the converged data architecture.
Use either JMS or Kafka APIs for messaging with Oracle Transactional Event Queues. The use of the standard APIs provides application portability and lets you seamlessly build hybrid and multi-cloud messaging applications.
Run applications on fully-managed services with built-in infrastructure maintenance and security patching. You can leverage scaling automation in response to the changing loads. Use Oracle Autonomous Database a fully managed Oracle Database service on OCI. Oracle Transactional Event Queues is a built-in feature of the database available in all OCI regions.
Use Oracle Autonomous Database that natively support different types of data—JSON, relational, graph, spatial, and so on. Use database functionality to simplify application logic. For example, use SQL for queries, joins, and analysis. Use transactions to guarantee consistency and isolation, and built-in machine learning algorithms and analytics to avoid unnecessary data transfers. Use the database’s security features and access control to protect sensitive data, and use replication to improve the availability, scalability, and resiliency of your app.
A messaging application can contain hundreds of services, owned by different application and business teams. Unified observability with open tool interfaces is important for gaining visibility into the behavior of these complex distributed systems. Instead of each team building their own solution, centralize the metrics, logs, and tracing exported from all layers of the application. TEQ metrics can be exported into Prometheus, which supports Grafana dashboards for debugging and performance tuning workflows.
Eliminate single points of failure through horizontal scaling, and automate failure recovery.
Since TEQ is a part of the Oracle Autonomous Database, applications benefit from the built-in High Availability and cross-region Disaster Recovery capabilities without any additional work.
TEQ’s transactional messaging also simplifies recovery from external failures.
Implement Identity and Access Control (IAM) policies to allow only authorized users to create, send, or receive data from the streams. Apply a principle of minimum reachability to the endpoint by securing access to messaging endpoints with mTLS and service gateway, which limits access from the Internet. Encrypt data at rest and in transit to achieve data confidentiality. Use Database Wallet for securing credentials of connections to the database.
The converged database architecture provides a design to accomplish messaging patterns in modern applications. This pattern uses Transactional Event Queues (TEQ) that is built-into the Oracle Autonomous Database.
This architecture provides simplicity by eliminating the need to utilize external streaming or queueing services and provides transactional messaging capabilities that simplify common microservices patterns.
Application architects should consider this architecture’s scalability, performance, and simplicity:
FedEx uses Oracle E-Business Suite and the business event manager built with Oracle TEQ, for accounts receivables of the 15.5 million packages delivered every day.
Several 2-factor authentication (2-FA) scenarios that use one-time passwords (OTP) are enabled by Oracle Transactional Event Queues
Learn more about related resources.
Low-code platforms are well suited for building opportunistic applications in collaboration with business stakeholders; building data reporting and analysis apps; extending SaaS apps; and modernizing legacy applications. Every line of code has a cost associated with it—to author it, maintain it, debug it, upgrade it, and secure it. Oracle Application Express (APEX) helps developers avoid these costs by providing high-level components and common design patterns through an intuitive and graphical development experience.
Low-code platforms enable you to build enterprise apps faster than with traditional hand-coding. These platforms are well suited for building opportunistic applications in collaboration with business stakeholders, building data reporting and analysis apps, extending SaaS apps, and modernizing legacy applications.
With low-code platforms, you are free to focus on solving your business problem rather than on the complexity of developing web applications. These complexities include security, accessibility, efficient data access, performance, and globalization. Low-code platforms eliminate this complexity by dramatically reducing the amount of code you have to maintain.
Oracle Application Express (APEX) helps developers avoid the costs associated with traditional app development by providing you with high-level components and common design patterns through an intuitive graphical development experience.
Oracle Cloud Infrastructure (OCI) provides the secure, reliable, scalable, and highly performing infrastructure required for the most demanding applications. These applications can be scaled to support anything from small workgroups to millions of end users. This document describes the design principles and optimal implementation path for architecting a low-code application.
When implementing a low-code pattern, use the following Modern App Development design principles:
Adopt a metadata-driven, low-code approach when developing apps. Specify application logic declaratively where possible, and write code only where necessary. Interact directly with data in the database by using SQL. Use fully-managed services, such as Oracle Autonomous Database and Oracle APEX Application Development (APEX Service), that can maximize availability and scalability to handle the changing demands of your low-code apps. In addition, database capabilities such as Oracle Real Application Clusters (RAC) and Oracle Data Guard ensure that your low-code apps are available 24/7/365 and can fail over if a failure occurs in the data center where the infrastructure is hosted.
Use the OCI Resource Manager to automate the provisioning of Oracle Autonomous Databases and APEX environments. Use Oracle SQL Developer Command Line (SQLcl) with Liquibase to automate the deployment of data model changes. Use APEX one-click application deployment to manually deploy changes between environments.
Oracle APEX is stateless and serverless and the runtime application state is stored in tables so connections can be re-used across users. This results in far fewer connections that can support high user concurrency.
Monitor and trace APEX application activity by using the built-in Activity Monitoring capabilities, which include detailed tracing and debug information on a per-user level. Review workspace-level and instance-level activity from APEX Administration Services. Use Performance Hub to monitor database activity, review automatic workload repository (AWR) reports to identify top resource consumers, and to identify tuning recommendations.
APEX on Oracle Autonomous Database is deployed using highly available architecture that encompasses the data tier (Exadata and RAC) and the middle tier (redundant Oracle REST Data Services nodes). Use Autonomous Data Guard to further increase the availability of your apps to protect against availability domain failures.
Use OCI Identity and Access Management (IAM) for authentication schemes for your APEX applications. Assign authorization schemes to APEX apps and app components to enforce access control based on user role or privilege. Use the built-in declarative capabilities of APEX to handle Session State Protection (SSP) and item-level encryption to protect your apps and data. Use bind variables in SQL queries to prevent against SQL injection. Configure app-appropriate timeouts to ensure that inactive sessions are automatically terminated. Run the built-in APEX Advisor to detect potential security concerns such as unprotected pages, items, and buttons. Use declarative escaping and programmatic escaping APIs to guard against cross-site scripting (XSS).
This architecture is for low-code app development and deployment using Oracle APEX Application Development (APEX Service) and all Oracle Autonomous Database services. By deploying these services, all the components necessary for the full stack are automatically provisioned and fully managed. These components include gateways, load balancers, and Oracle REST Data Services.
This image shows the architecture underlying low-code app development and deployment using Oracle APEX Application Development and all Oracle Autonomous Database services. It shows a fully-managed environment containing a public subnet and a private subnet. Outside the fully-managed environment are these service: APEX Applications, Oracle REST Data Services APIs, and External REST APIs. These services access the fully managed environment through either an Internet gateway or a NAT gateway.
The public subnet contains a load balancer while the private subnet contains an Oracle REST Data Services instance and an Oracle Autonomous Database, upon which is an instance of Oracle APEX on Autonomous Database.
Traffic from the APEX Applications and the Oracle REST Data Services APIs is routed through an Internet gateway to the load balancer, which directs it bidirectionally to the Oracle REST Data Services instance in the private subnet. This service, in turn, communicates bidirectionally with Oracle Autonomous Database. Traffic from the Oracle APEX on Autonomous Database instance is routed directly to the external REST APIs through a NAT gateway.
All application artifacts are deployed to the database tier, which provides zero-latency data access because there’s no need for network traffic between the application and database tiers.
When implementing a low-code pattern, you should also consider the following:
Consider the alternatives to the architecture described in this pattern and avoid attempting to implement antipatterns.
Some examples of the efficacy of a low-code pattern are:
Learn more about related resources.
Big data is a set of capabilities and patterns that enables you to manage, collect, store, catalog, prepare, process, and analyze all data types (unstructured, semi-structured, and structured)—whether they come from sources such as databases, videos, forms, documents, log files, web pages, or images. Further, a machine-learning platform should be fully managed and allow data engineers and data scientists to perform all these steps in the model development lifecycle.
Oracle's big data capabilities span various services and tools so that you can begin your big data journey based on your skills and preferences. With converged Oracle database variety of data is stored in the database, and scales up to multiple PBs of volume, with fast ingest of data, and events (with Transactional Event Queues), providing security of all data with built-in security. The converged data architecture addresses the volume, variety, velocity, and veracity of data in a converged database platform with Oracle Autonomous Database.
Data scientists and machine learning engineers don’t want to spend time provisioning, upgrading, patching, and securing infrastructure. They want to spend their time building, training and deploying models that impact the business. A machine learning platform should be fully managed and should allow them to perform all the steps in the model development lifecycle (build, train, deploy, and monitor). Data used for machine learning should be source agnostic and let data scientists access consistent and reliable data for building, training and deploying models.
Most modern machine learning toolkits are open source and written in Python. As such, a machine learning platform should provide native support for open source frameworks and Python. It should also give users the ability to customize their machine learning environments by either installing their own libraries or upgrading the ones already installed. The platform should let data scientists train their models on structured, unstructured, or semi-structured data while scaling the extract, transform, and load (ETL) or training steps vertically or horizontally across a number of compute resources.
Finally, a machine learning platform should ensure that models can be easily deployed for real-time consumption with minimal friction (ideally through a simple REST call), while preserving the lineage of the deployed model to ensure that it can be audited and reproduced.
When implementing a big data and analytics pattern, use the following design principles for Modern App Development.
Your data is only as valuable as your ability to use it. Big data tools are popular in the open source community, and most of the capabilities from them have equivalent capabilities in modern databases, especially with data warehousing, analytics and AI/ML model training and deployment. The converged Oracle Database is one such big data platform.
Oracle's native premium capabilities such as Oracle Autonomous Data Warehouse external tables and SQL allow the use of data lakehouses in the Oracle database and OCI object store for storing and analyzing petabytes of data in real time.
DataOps is important for ensuring that you can derive maximum benefits from your big data pipelines. Use the Oracle Cloud Infrastructure Data Integration service to ingest data, implement ETL processing and ELT pushdown, and create pipelines for connecting tasks in a sequence or in parallel to facilitate a process. Pipelines can include various popular data sources within and outside of Oracle Cloud. Use Data Integration scheduling capabilities to define when and how often to run each task. Use Oracle Database Cloud Service Management to define database jobs that run against a set of databases on a schedule. Enhance this with CI/CD pipelines and unified observability for app developers.
Use the best tools that can simplify, automate, and accelerate the consolidation of data for use for maximum business value. For data warehouses, departmental data marts, and serving and presentation layers with structured data, use Autonomous Database, which is optimized for these scenarios. Autonomous Data Warehouse capability also provides connectivity to analytics, business intelligence, and reporting tools like Oracle Analytics Cloud.
Plan to keep your data secure. Track all jobs that bring data in and take data out of your data lake, keep data lineage metadata, and ensure that access control policies are updated.
Follow the principle of least privilege, and ensure that users and service accounts have only the minimal privilege necessary to perform their tasks. Control who has access to the data platform components by using Oracle Cloud Infrastructure Identity and Access Management. Use multi-factor authentication in Oracle Cloud Infrastructure Identity and Access Management to enforce strong authentication for administrators. Use Database security with Oracle Datasafe to get security posture of all data, users, and access patterns. Store sensitive information such as passwords and authentication tokens a Vault service.
Use a scalable converged database (transaction processing and data warehouse) to operate the enterprise which generates data, stores and analyzes all types of data. In this architecture, various data sources (end users, devices, events, sensors, and applications) feed data to the database through data integration (Oracle GoldenGate) and Oracle Transactional Event Queues for streaming data. The data is stored in Oracle Autonomous Database (Oracle Autonomous Transaction Processing and Oracle Autonomous Data Warehouse) along with OCI Object Store support for big data using SQL with external tables. Use Oracle Machine Learning for model building and deployment, and use Oracle Analytics Cloud for insights into the data.
Oracle Machine Learning (OML Notebooks, OML AutoML UI, OML Services, OML4Py) on Oracle Autonomous Database for exploring and preparing data, and building, evaluating, and deploying machine learning models.
This architecture pattern provides powerful capabilities when the data required to train the model is brought into the database, and processing is close to where the data is. This pattern has a variety of data sources and events that can be explored and prepared with data exploration for OML training, and use OML to build, train and deploy models with SQL, Python using OML4Py, or using no-code AutoML. Import models trained elsewhere, like OCI Data Science (e.g. Tensorflow or PyTorch) with OML Services, using the ONNX model format.
This pattern utilizes the ‘move the algorithms to the data’ approach. All data is accessed at its source (using external tables) or ingested, and then processed and stored into converged database for ML models. Once the models are trained in the database, they can be directly deployed using SQL queries (using PREDICTION operators) or with OML4Py APIs. Externally trained models can be deployed using OML Services. Additionally, text, spatial, and graph analysis machine learning is available in the Autonomous Database.
This architecture uses the following components:
The following data sources are covered:
This architecture has the following components within the VCN:
Consider the following for big data and analytics.
When implementing machine learning and artificial intelligence, consider these options.
When designing an implementation, consider the following:
Consider the alternatives to the architecture described in this pattern and avoid attempting to implement antipatterns.
The following are example implementations that use Oracle Cloud Infrastructure (OCI) data and analytics services to ingest, store, catalog, prepare, process, and analyze big data.
Use Oracle Autonomous Data Warehouse as a data warehouse or data mart with Oracle Analytics Cloud.
Manage data in both Autonomous Data Warehouse, and use Oracle Analytics Cloud for visualization of the data.
Learn more about related resources.