An issue was identified with Instiller receiving workflow pushes for EPA incompletes. After other more critical priorities were addressed, the issue was looked into for resolution.
NOTE: There is an update to this.
The current process that gathers the incomplete EPA submissions to send to Instiller and trigger the workflows is entirely done within the flux-quote-service which is the microservice that processes and handles the quoting part of the EPA process.
An error was identified in the process which matches quote brand/types (know internally in Adrian Flux as “QAB types”) such as Flux/car or Flux/van. As new QAB types were added in the service, the send-to-instiller process could not handle the new types, and it exited he processes prematurely. A patch was issued to the quote service on Tuesday night (2023-06-13). This changed the process to new types without an Instiller workflow would just be ignored and the process would continue as usual. This would be available in the next process run at 03:00AM.
The workflows were checked on Wednesday morning, and the workflows received no updates as expected. Further investigation was needed as there were clearly issues further down the line that was affecting the data being sent and the workflows being triggered.
This is one of the older webservices and as such it is running on an older platform in AWS. It appears that the Instiller API / servers are not accepting the older protocol that the server is running on to send the requests. The quote service platform is scheduled to be updated within the next month, and this would fix the issues. However, we are keen to resolve the issue sooner. To do this we are going to create a handler on another service. This other service already sends other submissions from other sources to instiller. We will essentially move the process from the quote service to this other service. The quote service then just needs to pass the data onto this service and the workflows will trigger as expected. This other service will also have greater transparency of what is sent and when, which will help diagnose issues and be more proactive in its monitoring. It will also be possible to receive a daily email digest of what has been sent.
This work is scheduled to take place this Friday (2023-06-16) with deployment to the server happening on Saturday (2023-06-17) so the run should submit on Sunday 3AM for Saturdays incompletes. The hopefully successful resolution will be communicated on Monday.