Intro
With my professional background, I have a lot of experience troubleshooting and resolving technical problems in Boomi. I’ve spent thousands of hours poring over logs, scouring the Boomi Community site, and wading through Stack Overflow and other sources of information. Anyone who’s done the same knows very well that this task is often done on a time limit because you either have development deadlines or you’re detangling some live production crisis.
It’s for this reason that one of the newer Boomi AI agents, Boomi Answers, caught my attention. I didn’t expect it to be a troubleshooting tool to replace an experienced dev or a Support team – it was trained on the contents of the Boomi Community and the Help documentation, so the information available there was always going to be the natural ceiling of this AI’s usefulness. But could it distill a half hour of reading Community articles and user discussion down to an instant?
I fired up Boomi Answers on my account to dig in and see.
Boomi Answers advertises itself as a solution for three general topics: Platform Questions, Troubleshooting, and guidance in writing Groovy Scripts, so these three categories are where we’ll focus our tests.
Boomi Answers AI Agent for Platform Questions
This is the topic that I expected to be well within Answers’ wheelhouse. If you ask specific questions with distinct answers, it should be able to retrieve this data from the Community. I found that it did mostly return correct information, though not always. The data was sometimes incomplete, and on some occasions, it could even be entirely incorrect. It can also be a little temperamental and give different results depending on your precise wording.
First Question: What connectors don’t require a license to use?
First, I tried asking what connectors don’t take up a license, using several variations in the wording. The AI would always give me a link to the How Connection Licensing Works document, which is good, but oddly the response text itself would only return partial lists. It seemed to depend on my wording, with different queries returning different subsets of the full list.
Changing the question to “What connectors don’t count against your licensing” actually yields straight misinformation. See the very end of this response:
If you’re familiar with the topic, you may be pressing your palm to your forehead right now: custom connectors created with the Connector SDK absolutely do count against your licensing. It’s even documented right in the Community Document the bot links.
Second Question: Multiprocess and Forked Executions: how do they affect execution limits?
Next, I asked a question that is a little more advanced and that comes up on rare occasions when execution limits are being hit. Do multiprocess or forked executions count against your maximum simultaneous execution limit? Do they count against your Account Concurrent Execution limit on the cloud?
When asking the machine single questions, it tends to give the correct answer (which is “yes”):
However, when I tried to get fancy and ask questions with multiple factors, it began giving me different answers, even for synonymous prompts. Here is an example of conflicting information where the latter answer is correct but the first answer it gave is not:
Third Question: What does the Low_Latency_Debug option do?
Here is an example that showcases the need to be very specific in your questions. Low_Latency_Debug is a commonly-used HTTP Workload property in Cloud environments: it allows general-mode processes to run on a cloud’s Atom Workers. However, it appears that the AI tends to confuse the term with the more generic “low latency debug”; in other words, disabling low latency mode for single executions to force logging, to debug a low latency process. If you don’t specify that it’s a Workload property, you will get the wrong answer:
Boomi Answers AI Agent for Troubleshooting
First Question: Process Performance
The first question I chose for the Troubleshooting category was an old favorite: Integration Process performance. My prompt: Why has my process suddenly started taking longer?
There are a ton of possible reasons why a process might suddenly start taking longer to complete. And to the system’s credit, it came back with a good number of them!
The AI suggested some of the classics: increased load, higher system resource usage, and memory utilization, recent revisions to process logic or connector settings, and changes to external systems. When I modified the question to a general “Why is my process execution slow?”, it brought up an additional common reason: the use of the Flow Control shape.
This one was mostly successful. The only quibble I might have is that it can be long on generics and short on detail. It mentioned Flow Control shapes, but not specifically the “Run Documents Individually” option, which needs to be used sparingly. It gave me a link to the documentation on Low Latency Processes, but without explaining why. It mentioned system resources without specifically telling me to look for Low Memory warnings in the logs. It capped it off with “follow best practices” and “optimize process execution performance overall”, which are certainly laudable goals, if a bit light on specific action without further legwork. Boomi AI clearly needs to be just one element of a broader research plan.
Second Question: Multiple Head Nodes
Next, I tried a more specific problem, a pernicious issue that can afflict your molecule when network communication is blocked or constrained: Multiple Head Nodes.
To briefly summarize, one node of your molecule will be declared the Head Node, and it will focus on administrative tasks, work distribution, and maintenance. If the molecule doesn’t detect a head node, the nodes will talk to each other and agree on which one should take the role amongst themselves. If there is a network communication issue, this breaks down. An isolated node that can’t see the others would be forced to declare itself the head node, for example, resulting in MULTIPLE_HEAD_NODES.
This ended up being another situation where the specific wording of your prompt has an outsized effect on the quality of your answer. As you can see, two very similar prompts yield results of different qualities. The first answer is fixated entirely on one specific situation that can occur on startup, possibly because that one situation has a recent and highly-viewed Knowledge Base article at the time of this writing. The second answer also mentions this startup situation, but it at least also mentions that network communication issues can cause the problem:
If the article for the “startup scenario” continues to be heavily pushed by the AI for this problem, I would suspect that this situation will only exacerbate, as the article will rack up ever more views in a self-reinforcing loop.
Third Question: Enabling SSO for only some users
The AI came so close to getting this one right. You can see that it was on the right track about making some users admins, but it got confused and “reversed” the answer.
The documented method Boomi provides to enable SSO for only some of your users starts with having you enable SSO for everyone. Then, you get your list of folks who should still log in manually and make those users admins, as admins will retain the option to do so. The way the AI assembled the response seems to imply that you should make the users that need SSO admins, which is the precise opposite.
Boomi Answers AI Agent for Groovy Script Assistance
I had my doubts at first about how well the AI would do in providing scripting advice. While there were some issues, it usually did acceptably well at providing links with the guidance you’d need.
First Question: How do I write a groovy script that can set a dynamic document property?
This question returned one correct line of code but not a complete answer. It returned the setProperty command, but it failed to specify that I would have to import java.util.Properties for this to work.
It also linked me to two articles that weren’t really related: One on how to execute a Python Script and one about writing document properties to the process log. This is an odd oversight because there’s already an article in the community on how to set up any kind of property using Groovy that has complete examples for each readily available.
Asking the same question about dynamic process properties was a little better: This time, it actually mentioned needing to import the com.boomi.execution.ExecutionUtil class, in addition to the correct setDynamicProcessProperty line of code.
Oddly, it then linked the Javascript version of the article that directly addresses the question. There is a Groovy version available; I’m not sure why it chose the Javascript counterpart.
Second Question: Using Groovy for date manipulation
The result of this query is probably the ideal response we’d want to get back from an AI prompt. It gives a general overview of how to do it, provides the correct links that give you the specifics, and doesn’t try to give you any lines of code.
Third Question: Using Groovy to sort documentsThis result is technically correct, if a bit odd. It talks about a Knowledge Base article that will tell you how to accomplish this request. It then links an article that doesn’t contain that information, but which does link you to the article previously referenced. So the correct data is given. You just have to follow the rabbit hole down two steps to find it.
Conclusion: Is the Boomi Answers AI Agent Useful?
What we’ve found with this experiment is that Boomi Answers will generally return usable information, but you should always be aware that the information may not be complete and is not necessarily always correct. The answers will rarely be exhaustive, tending to yield high-level summaries that are short on specifics, especially if the question is multi-faceted or open-ended. AI is not truly intelligent; it cannot understand the actual content of the articles it’s pulling data from; it only assembles sentences based on patterns from training data. This is sometimes acutely illustrated in the answers the agent will return.
I would therefore advise the following strategies for best results with this tool:
- Ask a single, discrete question that should have a single answer where you can.
- Do not simply accept what the AI tells you or take its answer alone as definitive fact.
- Ask the same question twice with different wording and compare the results.
- Be sure to follow the documentation links it will provide to corroborate its claims. Follow up on the answers you receive, and confirm the results with other research!
In short, Boomi Answers is very much a handy tool to have in your toolbox, but should not be the only tool by any stretch. Follow the advice above, and it can be a useful step towards getting your Boomi questions answered.
For more information, please contact us!