Troubleshooting X3 Services issues (New ADC)

If you are having problems with X3 Services and want to diagnose the issue, what can you check and what information should you gather in preperation for logging a case with Sage X3 Support team?

1.System configuration

a. Confirm the system architecture. i.e. is the installation all on one server or multiple servers? If multiple servers, please provide details of which X3 services are on which servers

b. Confirm versions of Syracuse, Runtime, Folder and X3 Services

2. Provide the Syracuse "nodelocal.js" configuration file , plus the X3services "xtrem-config.yml" and "xtrem-security.yml" configuration files for review

3. Provide screen shot from the following functions within Sage X3:

a. Administration, Administration, Endpoints, MS SQL service

b. Administration, Administration, Endpoints, X3 solutions - showing the "Services" section

c. Administration, Administration, Web services, Classic SOAP pools configuration - for the list of web pools and also the detail for the ADC web pool

d. Administration, Administration, Servers, Hosts - showing the detail of the server

4. Confirm what device you are using to test. We would normally expect you to be using a browser from the Server console as the starting point.  In this case also confirm which browser(s) have been tested (Chrome/Edge/Firefox)

5.   Enable X3 services logging as described in KB article "Diagnosing "New ADC" (Automated Data Collection) issues (aka X3Services)" ( https://gb-kb.sage.com/portal/app/portlets/results/viewsolution.jsp?solutionid=230713124154953 )

6.  Increase Syracuse logging (temporarily)

a.  Login to Sage X3 as an administrator user

b.  Navigate to Administration, Administration, Settings, Global Settings
In the logs section, set the following parameters to ensure the log files can grow big enough for the logging needed, and the log files do not get removed too quickly.  These suggested settings assume you have the default values currently set:
-- Maximum file size (MB) set to "200" (10 default)
-- Maximum number of days "15"  ( 5 default)

c. In the "Server Logs" section, set the following sections to "Info" log level
--- http.in
--- http.out

d.  In the "Server Logs" section, set the following sections to "Debug" log level
--- console
--- memory
--- System.config
--- System.monitor
--- System.process
--- soap.classic
--- soap.request
--- soap.server
--- etna

NOTE: these changes take effect immediately when you save these changes.   Ensure you remember to reset these back to the original values, once the troubleshooting process for this issue has been completed.

7.  Reproduce the issue, making sure you take screen shots to demonstrate the steps needed to get to the error from logging in to the point of the error.  Make sure the date/time is shown in the screen shots for easier cross reference to the logs.

8. Gather the following information:

a.  Screen shots of the user actions

b.  X3 Services log (xtrem.server-XXX.log)

c.  SQL Server ERRORLOG for the date/time when the issue occured

d.  Syracuse log files 
Syracuse log files are located under the install directory, for example c:\Sage\Syracuse\syracuse\logs
Upload for review the "LB", "N" and "W" Syracuse process log files for the date concerned.  These have the filename format:
--- YYYY-MM-DD.SERVERNAME_LB.log
--- YYYY-MM-DD.SERVERNAME_N*.log
--- YYYY-MM-DD.SERVERNAME_W*.log
Also upload the "sage_syracuse_-_node0-stderr.YYYY-MM-DD.log" and "sage_syracuse_-_node0-stdout.YYYY-MM-DD.log" covering the date concerned

9.  Revert the additional logging changes

10.  If the issue seems to be with the AOWSIMPORT import stage, then you can take additional steps to diagnose:

a.  Revalidate appropriate import/export templates (AOWSIMPORT, possibly some of the CWS* ones as well)

b.  Republish AOWSIMPORT, CWSSCS, CWSSIS and CWSSST web services

c.  Generate Engine Trace for the import as described in KB article "How to run an Engine trace in Sage X3" ( https://gb-kb.sage.com/portal/app/portlets/results/viewsolution.jsp?solutionid=230724104552830 )
--- In Admin, Usage, Logs, X3 session logs select the appropriate Web Service AWSIMPORT and the SOAP pool being used for ADC. Set the log level to LOG_TRT1 (#2) and enable tracing. Then re-run the SOAP import and the log file gets written to the runtime\logs directory.

With the above information you may be able to spot the issue yourself, but if not you will have the basic information with which you can then log a case with the Sage X3 support team.

Diagnosing "New ADC" (Automated Data Collection) issues (aka X3Services)

Additional notes when installing or upgrading X3 services (New ADC)

How to run an Engine trace in Sage X3