Creating JDE orchestrations to import data

Among the many functions of JD Edwards EnterpriseOne Orchestrator Studio, the tool simplifies data exchange. In a world that increasingly expects devices to talk to one another, this can mean the difference between transformation and stagnation. We last looked at the process for exporting JDE data to a format preferred by your external application, system, or device. Now, let's look at bringing data into JDE with orchestration.  

The basics of importing data

REST services typically leverage JSON because the output notation is easy for people, applications, and web services to read. That being the case, JSON is the default input/output format for orchestration across all EnterpriseOne tools releases. JDE users running E1 9.2.2.4 up to 9.2.3.2 can write a custom Groovy script to convert XML or CSV to JSON before importing. But E1 9.2.3.3 users have it easy. The ability to import XML or CSV data is baked right in. 

While JSON remains the default input/output format, E1 9.2.5 introduced native support for importing data from Microsoft Excel files (.xlsx). This eliminates the need for complex workarounds using E1 Pages or VB scripts for smaller datasets.

In our exporting data blog, we explored how to set your custom HTTP headers and shared sample JSON cURL commands. These apply to data imports as well. However, the process changes when working with CSV. EnterpriseOne 9.2.3.4 and later are ideal for CSV orchestration because it allows multiple line entry. Oracle support has an excellent example in Doc ID 2564800.1.  

Security Considerations for Data Imports

As you leverage Orchestrator for data imports, it's crucial to consider security best practices:

• User Permissions: Ensure that users creating or running import orchestrations have the necessary permissions to access and modify the target JD Edwards application data.
• Data Validation: Implement data validation routines within your orchestrations to ensure the imported data adheres to the expected format and data integrity rules.
• Data Source Trust: Be cautious when importing data from untrusted sources. Consider implementing data cleansing or sanitization steps within the orchestration to mitigate potential security risks.

E1 Tools 9.2.3.4 orchestrations for importing data 

To create a CSV import orchestration in E1 9.2.3.4, your CSV should include an FTP connection and connector for your orchestration to read. Learn more about configuring the FTP connector/connection in Doc ID 2564800.1 and on learnjde.com. Then follow these three high-level steps:

1. First, create a service request to import the data into the application, for example, P4210. You can create the request manually or by using the Orchestrator Process Recorder.
2. Next, build an orchestration with generic JSON input. This way, you can test the basic service request flow before modifying the orchestration to handle CSV input.
3. Finally, create a service request for an FTP connector. Then, configure a transformation to iterate over the lines/records in the CSV file you have mapped.

With template(s) in place and a bit of practice, the process becomes relatively straightforward. E1 9.2.x and later offer improved CSV import capabilities. Orchestrations can now handle larger files with more than 10,000 rows by leveraging increased JAS JVM heap allocation.

In E1 tools 9.2.6.x and later you have additional options using scripting tools (Jython, Groovy or JRuby) to manipulate the data to your needs. Oracle support Doc ID 2905923.1 has several good examples to reference.

Most JDE users are comfortable using an FTP site to retrieve the CSV file. However, advanced users with HTML coding expertise may choose to create an E1 Page with JavaScript to read CSV or an Excel input file. The E1 Page then passes the data into the orchestration. It's more complicated but beneficial for low numbers of records between one and 1,000 rows of data. You can also use Microsoft Excel with a VB script to call the orchestration to load the data.

Orchestration tips for importing data

• Remember, orchestrations operate with the same system/architecture settings as web users. Therefore, a CSV input orchestration is similar to a spreadsheet grid import. The JAS grid import is limited to 1,000 rows by default. But with enough JAS JVM heap, you can increase it to 10,000 rows. 
• If your application only allows 1,000 rows of input, use multiple sets of entry. Application limits are one reason some architectures separate the AIS/JAS instances on a dedicated server depending on business requirements/processes to provide more resources.
• Change the JAS instance Maximum Request Parameters default to ensure your record count is not limited. We generally set the parameter to 500,000. Learn more. 
• Set the AIS and JAS instance Java heap min/max size. A good starting point for the AIS instance is between 256m and 1024m. The JAS instance used by AIS for orchestrations varies from 1024m to 6192m. We set most configurations between 1024m and 2048m then monitor the JVM heap usage. However, in some large use cases AIS heap as large as 20 GB has been used.
• In AIS there are also disk cache and compression settings similar to JAS that you can enable to help with larger requests

JSON may be the standard for inputting data, but it isn't your only option. Get familiar. Get creative. Use a combination of input and output notations to automate data flow and connect your digital ecosystem. 

Unlock Potential

JD Edwards Orchestrator is a powerful tool for automating tasks, integrating systems, and streamlining workflows, helping you unlock a new level of efficiency and productivity. Get in touch with us to find out how we can be your partner in all things Orchestrator.