Modernization Deep Dive: Connecting ERP & CRM for Better Customer Visibility

What happens when you remove middleware and let JD Edwards and Salesforce communicate directly? In this Modernization Deep Dive, Gary shares a real-world integration project that replaced traditional middleware with direct, real-time API connections. See how a four-person team built 22 integrations in six months, reduced latency and sync errors, and created a more scalable approach that keeps critical data in sync across JD Edwards and Salesforce.

Transcript

Today we’ve got our second-to-last session, and this one is on connecting ERP and CRM. Specifically, today we’re going to talk about Salesforce integration with JD Edwards.

We’ve got our next two hosts here. Gary Crowell is with us today. He’s our Senior Technical Developer. Also with us is Kevin Van Horn. If you’ve been on any of the previous sessions, you’ve probably already met both of them. They’ll help present and handle any questions.

I’m Scott Volpenhein, Marketing Director here at ERP Suites.

As with our previous sessions, we’ll record this. If you have questions, certainly ask them using the chat. We’ll get to them during the presentation and at the end if we don’t get to them live.

Before we get going, I do want to give a little background about ERP Suites. We do everything with JD Edwards around cloud hosting, managed services, AI, and consulting, whether it be functional or technical consulting. We also do EPM services with Mike Pilato here on our team.

Specifically with AI, it’s not just JD Edwards. We do enterprise AI. We’ve got Salesforce that we’re talking about today, but it could be Salesforce or anything your business uses and needs to connect to. Our AI platform is built to handle that, especially with agents.

I’ll turn it over to Gary to go over the agenda and kick off our presentation.

One thing I will say: today is a little different from our previous sessions. We don’t have a demo. Gary is going to go over everything we did for this project, specifically connecting the CRM. There’s no actual demo, but we have a lot of slides and we’re going to tell that story.

Go ahead, Gary.

Thanks, Scott. Hi, I’m Gary. As Scott said, I’m a Senior Technical Developer here.

What this really is is the story of what we had, what we did, the challenges we faced, and how we accomplished it.

For the agenda, we’ll talk about the before state, the recurring cost, and how it was done. Then we’ll cover the decision, the approach, the architecture, and the actual build.

We’ll also cover push versus pull if you’re not familiar with that terminology, along with master data challenges, large data sets, and the payoff. Hopefully, you’ll have questions at the end that we’ll take.

With that being said, we’ll get started.

The short version of what we walked away with was $150,000 a year in savings, with an asterisk. If you know anything about middleware, most platforms charge you per transaction. As your business does better and your transaction volume increases, your costs increase.

A lot of middleware also relies heavily on pulls, polling, and querying for data that may or may not be there.

For master data specifically, we went from polling every two hours—with some polls taking 15 minutes—to updating master data in real time in less than three seconds.

That includes the time for the systems to go through the gateway, talk to each other, validate credentials, run the transaction, send it back, complete the handshake, and return any errors.

We also saw an estimated 75% reduction in integration failures and sync errors.

Basically, there’s no middleware and no nightly batch window. If something breaks, there’s no integration vendor to call and no third party that has to get involved.

This was a fairly large project, and we did it with four people: a Salesforce developer on staff, two JD Edwards developers—myself as the technical lead and developer, along with another developer—and a business user from the client.

That business user was really the critical element that made this successful. We couldn’t have done it without her.

Four people. Six months. And 22 integrations that are running today.

We completed this about nine months ago, and I haven’t had one call about it. Everything is running smoothly.

We’ll look at the before state, what we did, and what it ultimately became. Think of it as act one, act two, and act three of the story.

Before this project, these were essentially two systems that couldn’t have the conversations they needed to have.

Sales lived in Salesforce. Everything else was in JD Edwards.

When sales reps were answering questions or looking at prices, the information was only as good as the day before. If something changed, they wouldn’t receive the updated information until the next day when the system polled.

They also couldn’t see real-time order status in Salesforce. They would have to bounce between two systems to get answers.

The same was true for credit holds. They couldn’t tell what was happening with a sales order directly from Salesforce. If an order went on hold, they wouldn’t know it without going into JD Edwards and looking.

They couldn’t see everything from one place.

Any changes also meant going through the middleware change request queue. It was cumbersome, slow, and out of sync with a fast-moving environment where a customer service representative needed answers immediately.

Other than sending the sales order transaction over, the integration almost became pointless because users still had to bounce between two systems.

The data was always right. It just wasn’t right now.

It was always a day late and a dollar short, so to speak.

If you’re dealing with middleware, you know your data is only as good as the last poll. That might happen a couple of times a day or sometimes only nightly. It isn’t truly real time.

What are you paying for with middleware?

Some of it you pay for with money. Other parts you pay for with inconvenience, time, latency, and complexity.

The biggest cost that shows up is licensing. As your business grows, you need more connections. As your transaction volume grows, you have to increase your capacity and costs because you’re paying per transaction.

There’s also latency by design. Your data might be four hours old or sometimes a day old, depending on when the poll ran.

If something goes wrong, you’re no longer dealing with two systems. Now you’re dealing with three.

Then there’s the skill set. You don’t necessarily own that skill set internally. A third-party vendor owns it, and you have to call them.

It becomes much harder to support.

Those challenges are prevalent in many middleware applications used in scenarios like this.

How did we get here?

The company had a new CIO who essentially asked, “Why are we doing this?”

It stemmed from a meeting with the middleware company because the business wanted to do more. They knew transactions were increasing, and they were already reaching their volume maximums.

The answer they kept getting was more time and more money.

The CIO’s response was, “There’s got to be a better way than this.”

Given where technology is today, the existing approach felt outdated. There had to be something better.

Both platforms already used REST.

The Salesforce platform was already using integrated sales APIs. The API was simply going to the middleware instead of going directly to JD Edwards.

Authentication was already done. Everything was there.

The translator in the middle was solving a problem that no longer existed.

The two systems could talk to each other.

That’s what led to this project. Why do we need something sitting between these two systems when both already have REST capabilities and can communicate directly?

That question drove the project and the goal of making the integration more streamlined, cost-effective, real time, scalable, and stable.

This is an important part of the project because we established three rules before writing any code.

The first was point-to-point: direct API calls in both directions.

Some integrators, even when using APIs, still rely on pulls and look for data whether it’s there or not. We chose to go both ways. It was going to be point-to-point and push-push.

If we needed a third system to be involved, we were doing it wrong.

The second rule, which was really the game changer, was push on a change, never on a schedule.

What we mean by that is if I change something relevant in Salesforce, I push it immediately to EnterpriseOne.

Typically, that’s a transaction coming from Salesforce. If I enter a sales order, I want it to go immediately to E1, and then I want the response to come immediately back.

On the other side, if something changes in JD Edwards—an address book record, customer master line of business, price, phone number, or email address—we don’t want to wait.

We want it pushed when it changes.

“I changed. Here you go. Here’s the data.”

Now the systems are in sync.

The third rule was every endpoint is a contract. It’s versioned and documented.

Orchestrations are products. They aren’t scripts someone wrote once. They’re scalable products that can be designed so that as your business changes, you don’t necessarily have to call us or another partner to make a change.

You can work with your data and configuration without turning every business change into another development project.

But rule two is really the one that changed the game because you’re no longer waiting on last night’s sync.

What happens if last night’s sync didn’t run? Now you’re two days behind.

With this approach, we’re getting real-time changes, real-time information, and real-time transactions.

That’s the biggest takeaway: everything is real time.

The customer service representatives can live in one system and see everything from the system they use every day.

All of the customer service reps lived in Salesforce. The fact that they had to go into JD Edwards to look up something that Salesforce should already know was cumbersome and didn’t make sense.

With the new architecture, three hops became one.

Before, it was Salesforce to the middleware platform to JD Edwards.

Salesforce would ask the middleware for something, the middleware would poll for it, and then populate the information back into Salesforce.

With the new system, you scale it down to two systems—the two systems that actually care about each other.

In this instance, the middleware had become noise.

When you look at it this way, the question becomes pretty simple: Why do I need that middleware?

Everything was already in place to operate without it.

When it came to the build, everything was a push.

The system that owns the change announces it.

Essentially, it says, “Hey, I changed something. I’m letting you know about it.”

A couple of examples are sales orders. The minute someone enters a sales order, it goes straight to E1 as a sales order.

There are no Z files or anything like that. It’s a real-time transaction.

It comes back with the sales order number and any relevant information Salesforce needs, and it’s upserted at that point.

The same applies to changes to orders.

Previously, once an order was in JD Edwards, a customer service representative would have to go into JDE to make a change and then wait for those order changes to come back to Salesforce during another nightly run.

New customer requests raised by sales had a cumbersome process as well.

Now they can enter a new customer, send it over, and it gets uploaded to JD Edwards.

Going from JD Edwards to Salesforce, most of what we’re talking about is customer-related data or pricing data.

It’s master data, and it’s kept in JD Edwards.

The way the integration was originally designed was to continuously poll for information whether or not anything had changed.

For example, you might get daily polls for customer information.

But customer information doesn’t change that often. A company doesn’t change its address every day.

When something does change, you need to know about it. But you don’t want to burn network time, resources, hardware, and transactions polling for something that hasn’t changed.

You could have four polls a day with no result.

Pricing worked the same way. Pricing might change monthly or at some other frequency, but the system would poll every day for pricing.

Pricing changes, contracts, item master changes—there was all of this noise for no result.

And when you’re paying per call, all of those polls that returned nothing still resulted in a cost.

One important distinction is what we deleted, not replaced.

There are no scheduled queries, no polling, no nightly extracts, and no reconciliation jobs.

The systems are in sync 24/7 with real-time information.

When I go into Salesforce and look at something, I know it matches what’s in JD Edwards because the two systems are talking directly to each other.

It isn’t a poll. It isn’t grabbing data, trying to massage it, make sense of it, and then determine whether it represents the same thing in both systems.

Now let’s talk about the push and the writes that have to stick.

Here’s basically how it flows.

An event fires in Salesforce. Queueable Apex—and Apex is the language Salesforce uses—picks it up and calls a JD Edwards orchestration.

Credentials and authentication are handled, JD Edwards validates the request, and the orchestration runs.

Here’s one of the biggest takeaways.

Because we’re using orchestrations and native JD Edwards applications and business functions, the transaction runs real business logic, just like a user sitting at the system.

That means real errors are caught.

There isn’t middleware sitting in the middle, pulling data, trying to make sense of it, populating it somewhere else, and potentially breaking business rules that may or may not have been correctly represented in the middleware.

Once the transaction is completed, we write the JD Edwards key back to the Salesforce record.

No key, no success.

If there’s no key, the transaction goes into a retry queue.

Salesforce calls it, and if for whatever reason it can’t get a response back, it goes into the queue and calls itself again up to five times.

If it still can’t complete after that, there’s probably an actual issue that needs attention.

At that point, it drops into an application on the Salesforce side that essentially says, “Hey, I could not make this transaction.”

Now you can act on that small percentage of exceptions.

Instead of spending all day wondering, “Is this data right? Is it not right?” you know exactly which transaction didn’t happen.

Then you can investigate it, manually enter it if necessary, or take whatever corrective action is appropriate.

Going from JD Edwards to Salesforce, the key is master data the moment it moves.

Here’s how that works.

E1 commits the change, the event fires, Salesforce upserts the record, Salesforce returns the result, and E1 logs it.

With a Salesforce upsert, if the record already exists, it updates it. If it doesn’t exist, it inserts it.

Anything that doesn’t successfully land is queued and replayed, not rediscovered.

Both sides have a handshake and an answer confirming what happened and providing the appropriate key.

That might be a sales order number, a unique ID coming from Salesforce to JD Edwards, an address number, or another relevant identifier.

There’s also no consequence to safely sending a record again. It’s going to update it or insert it appropriately either way.

For the master data, we used the tools already available to us.

On the OK button click, we used a form extension that called an orchestration and sent the record over.

We didn’t have to modify the applications. These aren’t customizations; they’re UDOs.

After the record commits, the form extension grabs the appropriate information and sends it over.

Any time a user presses OK and makes a change—or presses OK on a new record—the upsert in the CRM either inserts or updates that record.

That’s what makes the process safe.

If something gets pushed twice or repushed, it simply updates the existing record. You don’t end up with duplicate keys or duplicate records.

That was a big piece of this project because, previously, the customer service representatives didn’t feel like they could trust what they were seeing in Salesforce.

They didn’t know how old or new the information was, and they didn’t have visibility into whether a sync worked.

They simply had data.

So they were constantly keeping a Salesforce screen open, keeping a JD Edwards screen open, and toggling between the two systems to figure out the answer.

The answer should have been right in front of them in the system they use every day.

That’s what we were able to accomplish here.

Real-time data needs one owner.

The basic principle is: If I own it, I send it when it changes, and you accept it.

Those responsibilities are defined before you build the integration.

The owner of the data sends it when it changes.

Again, it’s a shift in mindset.

Instead of saying, “I’m going to look for something that may or may not be there,” the system says, “I’m giving you the information when you need it, only when you need it, right when it happens.”

Now we get to the hard part.

Real time is easy until the change set is 4,000 records.

The first rule here is that we only send deltas, never a full refresh.

You have an initial data set or seed as part of the cutover, but after that, you only send things that changed or new records.

You don’t send the entire customer record set every time.

The second consideration is batching the burst.

We identified certain processes where we knew that when something happened, there would be a lot of work to do.

Shipment information was one example.

Instead of continuously asking, “Did you ship anything? What’s my shipment status? Did you ship anything?” and repeatedly calling out and searching through different files, we took a different approach.

JD Edwards already knows when ship confirm happens.

This customer was on a batch ship confirm schedule, so we knew when that process ran and when everything had just shipped.

Immediately after that, we did all of the heavy lifting in E1 before ever making the transaction call.

That’s important because when you get into large data sets and polling, you can run into gateway timeout windows.

Instead, we scheduled a job immediately after ship confirm. It picked up the information, did all of the heavy lifting, and then called an orchestration and created the JSON output.

The actual API transaction could then be sent over in about a second.

Something that might have required 10, 15, or 20 minutes of processing took roughly a second of API time.

Salesforce doesn’t care about all of the processing that happened beforehand. It simply sees the inbound information, takes it, and it’s done.

We also designed everything around the smallest ceilings.

API limits, payload sizes, timeouts, governor caps—they all differ.

We built to the tightest constraint rather than trying to negotiate around it.

When we knew we couldn’t meet that standard, we took an alternate route, such as batching the burst.

If the target system is down, events queue and replay in order when it returns.

There’s no scanning, no job hunting, no drift, and no wondering, “Did this go through or not?”

It keeps trying until it reaches the system.

There can still be issues and errors, and we’ll talk about those, but this approach is significantly different from constantly wondering whether your polls ran and whether your data is correct.

Now, if I change a price in E1, it immediately gets sent over and is reflected in Salesforce in real time.

That creates a different level of confidence for customer service representatives.

They know they can look in Salesforce and trust that it’s the right information without having to check another system.

You can’t necessarily put a monetary price on that, but if you talk to the customer service representatives, they’ll tell you it’s invaluable.

It’s a game changer for their day-to-day operations.

When you’re doing something like this, you have to understand the process.

For example, when someone enters a sales order, that’s one transaction. It comes over instantly.

But if you want to pull a customer’s sales history for the last year, that’s different. JD Edwards can do the heavy lifting on its side, prepare the data, and then send it over. Once we actually call the API, it can still be less than a second for the transaction to go to Salesforce.

You have to analyze your transactions, your processes, and the data sets associated with each one.

That’s critical because if you use the wrong data set for a real-time API where you’re calling and waiting for a response, that’s where you can run into issues. You may be trying to do something in real time that shouldn’t be done that way.

There has to be a general understanding of what the data set will look like, what you’re asking for, and how long it takes to retrieve it.

One thing that caught us by surprise and wasn’t originally scoped was attachments.

I’ll caveat this by saying the client was on a 9.2.4 Tools release, so the situation would be a little different if you were on a later Tools release.

The way this client used attachments was important because every communication they had via email regarding a call was stored as an attachment of some sort.

With 9.2.4 Tools, there were some caps and limitations.

Long story short, we had to break the attachments into chunks. This goes back to understanding the limits and the size of what you can send.

On later Tools releases, there’s an AIS object called an Attachment Object. You can use that object to create an attachment on a sales order or another type of attachment used in JD Edwards.

It makes the process much easier on later Tools releases.

If you’re considering an integration like this and you’re not on a later Tools release, that’s something to consider. Newer Tools releases continue to introduce AIS objects that can make projects like this significantly easier.

Removing the middleware also removed our safety net.

The message queue we deleted had been handling errors, so we had to intentionally rebuild that functionality.

We did that on both sides.

If Salesforce sent a call to JD Edwards, it was logged and tracked as either a success or a failure.

The CRM supervisors or whoever was responsible for managing exceptions had a dashboard where they could see all of the successes, all of the failures, and detailed information about why something failed.

We did the same thing on the JD Edwards side.

We created a custom application and custom table where we logged all of the transactions coming in from Salesforce with either a success or failure flag.

When there was a failure, users had detailed information about what went wrong.

The entire payload was available so you could see exactly what came in, along with specific messaging showing where the transaction failed.

Because we were using native business functions, native applications, and native system functions, the errors were real errors coming directly from the system.

For example, if you were sending in a sales order and had a duplicate customer PO, that’s a real JD Edwards error. That error would bubble up and become the reason the transaction failed.

Maybe there was a date error. Maybe there was an invalid item. Whatever the issue was, these were real errors coming directly from the system.

That meant they could be handled immediately. You could fix the issue and allow the transaction to go through.

That’s something you have to plan for when building an integration like this.

To recap, we estimated the cost savings at approximately $150,000 annually.

In reality, it could have been even more because the company was ramping up and had already reached its transaction maximums.

There was also virtually no latency.

Take the nightly ship confirm process as an example. There’s technically no latency because nothing has shipped until ship confirm happens.

The minute something is ship confirmed, Salesforce knows about it. The team has the shipment information, ship date, ship quantity, and everything else they need as soon as it happens.

If they were using an interactive ship confirm process, we could have handled it similarly to the master data process.

Essentially, anything that happened could be communicated in real time.

At the time we put this together, the integration was averaging more than 20,000 API calls per month in both directions with a 99% or higher success rate.

If you’re dealing with the same problem or considering something similar, the first thing to do is check what your endpoints already do.

When we first got into this project, one of the first things I noticed was that there were already APIs that could be called from the CRM system, and we could create APIs in JD Edwards.

So the question became: What is the middleware actually doing?

The second takeaway is to call business logic, not tables.

The middleware relied on complex SQL statements to pull data. Then it had to massage and transform that data and make business decisions about what the data meant.

All of that logic was tied up in the middleware.

If the environment changed or the business rules changed, the middleware had to change.

With this approach, as the business rules change, the native applications and business functions handle those changes, and the appropriate errors come back through the system.

The third major takeaway is that you have to decide who owns what before you write any code.

Every integration argument is really an ownership argument.

Who owns the data?

Once you establish that, the principle is simple: the owner sends it.

You also need to think about the things that may not immediately come to mind, especially attachments and error handling.

Treat those as features, not details.

It’s easy to focus on the sales transaction, purchase order transaction, or another primary process and forget about things like header attachments or schematic attachments associated with parts.

You need to budget time for those requirements.

You also need to account for error handling because no matter how good your system is, you’re going to have errors.

If you were entering a sales order manually in P4210 or P42101, at some point you’re going to encounter an error on a sales order.

The difference here is that the errors are real.

They bubble up directly from JD Edwards. You simply have to make those errors available somewhere the user can see them and take action.

Because they’re real JD Edwards errors, they’re easier to act on. It’s similar to getting a red error while entering an order in P4210—you know there’s an issue that needs to be fixed before you can proceed.

There aren’t red herrings caused by middleware misinterpreting an error.

The whole project can really be summed up this way:

A small team using technology they already owned replaced a platform nobody thought was optional.

And when I say a small team, there were literally four of us.

There was one business user, one Salesforce developer, and two EnterpriseOne developers.

That was the entire team.

We completed the project in six months from inception to go-live, including post-go-live care.

This was a great example for a Modernization Deep Dive because you had two older processes and connections that could now be modernized using newer JD Edwards and Salesforce capabilities.

And if you talk to the customer today, it’s still a game changer.

The biggest thing is trust in the data.

If you’re polling and pulling data on some type of frequency, the information is outdated the minute you pull it until the next time you pull it.

During that interval, you don’t know whether something has changed.

That was why users were constantly bouncing between two systems and comparing information when they shouldn’t have needed to.

The reason this entire project came about was because the new CIO asked a simple question:

“Why are we doing this? Surely there’s a better way.”

And ultimately, there was.

Thanks, Gary.

If anybody has any questions, you can put them in the chat and we’ll try to cover them.

We have one more Modernization Deep Dive session in two weeks. We switched our last two sessions from Tuesday to Wednesday, so we’ll be back Wednesday at noon to talk about the Trial Balance Agent.

If you or anyone from your team is going to INFOCUS, we also have a lot happening there.

We have AI Agent demo slots available on Monday and meetings available on Tuesday. You can find those on our website by visiting our INFOCUS page.

Our rooftop party is Monday night, and Kevin will be debuting our Homebuilding AI Agents at INFOCUS on Monday at 12:45.

We’ll have more sessions on Tuesday, along with an artist at our booth, and our giveaway will be Wednesday at the end of the show. We’re giving away Ray-Ban Meta AI glasses, so stop by our booth, attend our sessions, and enter the raffle.

The only other thing I would add is that we’ve done integrations like this with other CRMs beyond Salesforce, and the same principles apply.

Thank you, Gary, and thanks to everyone for joining us.

Hopefully, we’ll see you in Denver.