Oresund Integration Platform
  • 🌉Overview of OIP
  • About
    • Functional Capabilities
    • Product & Developer Guide
      • Adapters
        • Rest Adapter
        • File Adapter
          • FTP Handler
          • SFTP Handler
          • EXCEL FTP & SFTP Handler
        • Database Adapter
          • DB Insert Handler
          • DB Retrieve Handler
        • SOAP Adapter
        • Quick Books Adapter
        • Mail Adapter
        • Subloop Helper
        • Global Variable Adapter
      • Monitoring Helper Service
        • Manual Monitoring
          • FTP/SFTP FLOW
          • API FLOW
          • DATABASE FLOW
          • General Monitoring
        • Scheduled / Automated Monitoring
          • Stopped job alert
          • File source Alert
      • Configurations & Business Specific terminologies
        • Business Specific terminologies
        • Configurations
          • Client & Client Setting configuration
          • Job & Job Setting Configuration
          • Step & Step Setting Configuration
          • Step Request Validator Configuration
          • Error Pattern Master Configuration
          • Failure Mail Configuration
      • Sample Description of I-Flows
        • API - API IFlow
        • API - FILE IFlow
        • DB - API IFlow
    • Dependencies
    • Product Roadmap
      • Summary of Features released / completed POC
        • New features planned for August 2023 - December 2023
Powered by GitBook
On this page
  • Step History Search Microservice
  • Functionality
  • Limitations
  1. About

Dependencies

PreviousDB - API IFlowNextProduct Roadmap

Last updated 1 year ago

Step History Search Microservice

Øresund Integration Platform (ØIP) uses Step History Search Microservice to store all the step-wise data generated during the execution. It uses open search / elastic search internally to store the data. The advantage of the implementation is as follows, OpenSearch is a fork of Elasticsearch that is designed to be open-source and community-driven while maintaining compatibility with the Elasticsearch API. There are several reasons why OpenSearch is implemented in this project. Efficient search capabilities: OpenSearch provides full-text search functionality that can be used to quickly and accurately search for specific information within large datasets.

Scalability: OpenSearch is designed to be highly scalable and can easily handle large amounts of data, making it ideal for systems that require high levels of throughput and data processing. Open source: OpenSearch is open-source software, meaning that it can be easily customized and integrated into existing systems without incurring significant licensing costs.

Functionality

OIP has a logging-enabled option that, when present, allows the step history data to be stored in OpenSearch. This process is achieved by calling the micro-service layer, which operates outside of the OIP.

At the runtime, the system can be dynamically configured to use either ElasticSearch or OpenSearch, depending on the requirement. Once this configuration is set, the micro-service takes over and performs the necessary computations. It also handles the Create, Read, Update, and Delete (CRUD) operations in OpenSearch or ElasticSearch, as per the configuration.

Therefore, the logging-enabled option, along with the micro-service layer, provides a flexible and efficient solution for storing step history data in OpenSearch or ElasticSearch.

Limitations

  • Can fetch only 10,000 records at a time in UI(this is the standard Open Search/Elasticsearch limit) however at certain times, We can temporarily increase the count and download the responses as excel and give it back. This 10,000 record is applicable for each filter we are applying.

  • We need to provide exact or near-exact search criteria for displaying correct records.

  • We need to delete records daily for the past 20 days by scheduling it as an I-flow.