Jamf Pro server is eating CPU after a while it breaks.

[ThijsX]

Hi,

After i have uploaded new MS Office packages for patch management, like i do many times our Jamf Pro Server JSS is eating 100% CPU and is not working properly.

No changes made, except above.
No recon overkill oi.

A fresh reboot will work for a couple of hours, but then.
Hosten on a VM on VEAAM, plenty of resources on Windows server 2016 distri.

Anyone seen anything like below?

2018-02-15 15:20:02,772 [WARN ] [Tomcat-298 ] [SqlExceptionHelper       ] - SQL Error: 1406, SQLState: 22001 2018-02-15 15:20:02,772 [ERROR] [Tomcat-298 ] [SqlExceptionHelper       ] - Data too long for column 'version' at row 1 Query is: insert into available_software_updates (computer_id, display_name, name, recommended, restart_required, version) values (?, ?, ?, ?, ?, ?) Query is: insert into available_software_updates (computer_id, display_name, name, recommended, restart_required, version) values (?, ?, ?, ?, ?, ?) 2018-02-15 15:20:02,772 [ERROR] [Tomcat-298 ] [lientCommunicationServlet] - Error processing communication content - org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; nested exception is org.hibernate.exception.DataException: could not execute statement 2018-02-15 15:20:10,585 [WARN ] [Tomcat-275 ] [SqlExceptionHelper       ] - SQL Error: 1406, SQLState: 22001 2018-02-15 15:20:10,585 [ERROR] [Tomcat-275 ] [SqlExceptionHelper       ] - Data too long for column 'version' at row 1 Query is: insert into available_software_updates (computer_id, display_name, name, recommended, restart_required, version) values (?, ?, ?, ?, ?, ?) Query is: insert into available_software_updates (computer_id, display_name, name, recommended, restart_required, version) values (?, ?, ?, ?, ?, ?) 2018-02-15 15:20:10,585 [ERROR] [Tomcat-275 ] [lientCommunicationServlet] - Error processing communication content - org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; nested exception is org.hibernate.exception.DataException: could not execute statement 2018-02-15 15:20:58,262 [INFO ] [duledPool-5] [VppLicenseMonitor        ] - Running license monitor 2018-02-15 15:21:03,606 [WARN ] [Tomcat-320 ] [SqlExceptionHelper       ] - SQL Error: 1406, SQLState: 22001 2018-02-15 15:21:03,606 [ERROR] [Tomcat-320 ] [SqlExceptionHelper       ] - Data too long for column 'version' at row 1 Query is: insert into available_software_updates (computer_id, display_name, name, recommended, restart_required, version) values (?, ?, ?, ?, ?, ?) Query is: insert into available_software_updates (computer_id, display_name, name, recommended, restart_required, version) values (?, ?, ?, ?, ?, ?) 2018-02-15 15:21:03,606 [ERROR] [Tomcat-320 ] [lientCommunicationServlet] - Error processing communication content - org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; nested exception is org.hibernate.exception.DataException: could not execute statement 2018-02-15 15:21:04,137 [ERROR] [ralPool-174] [ComputerChangeRunnable   ] - [Patch] Error while handling computer change java.lang.NullPointerException   at com.jamfsoftware.jss.patch.calculations.PatchCalculationService.isDeviceInPolicyLimitationScope(PatchCalculationService.java:135)   at com.jamfsoftware.jss.patch.calculations.monitor.patchRunnable.ComputerChangeRunnable.determinePatchPolicyLimitationScope(ComputerChangeRunnable.java:202)   at com.jamfsoftware.jss.patch.calculations.monitor.patchRunnable.ComputerChangeRunnable.handlePatchPolicies(ComputerChangeRunnable.java:143)   at com.jamfsoftware.jss.patch.calculations.monitor.patchRunnable.ComputerChangeRunnable.handleComputerChange(ComputerChangeRunnable.java:130)   at com.jamfsoftware.jss.patch.calculations.monitor.patchRunnable.ComputerChangeRunnable.run(ComputerChangeRunnable.java:76)   at com.jamfsoftware.jss.patch.calculations.monitor.patchRunnable.PatchRunnableWrapper.run(PatchRunnableWrapper.java:32)   at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)   at java.util.concurrent.FutureTask.run(Unknown Source)   at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)   at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)   at java.lang.Thread.run(Unknown Source) 2018-02-15 15:21:10,122 [INFO ] [duledPool-5] [VppLicenseMonitor        ] - License monitor completed after 11.86 seconds 2018-02-15 15:22:10,138 [INFO ] [duledPool-5] [VppLicenseMonitor        ] - Running license monitor 2018-02-15 15:22:14,169 [INFO ] [duledPool-5] [VppLicenseMonitor        ] - License monitor completed after 4.03 seconds

Cheers,
Thijs

[ChristopherGlov]

did you validate your checksum?

[ThijsX]

@beastie You mean for the uploaded MS packages?

EDIT: beside, thanks for respondig!

[ChristopherGlov]

@txhaflaire Yep

[ChristopherGlov]

You may want to take a look at the database tables. I underlined the string that stands out to me
From the logs it looks like a database issue:

2018-02-15 15:20:10,585 [WARN ] [Tomcat-275 ] [SqlExceptionHelper       ] - SQL Error: 1406, SQLState: 22001
2018-02-15 15:20:10,585 [ERROR] [Tomcat-275 ] [SqlExceptionHelper       ] - Data too long for column 'version' at row 1
Query is: insert into available_software_updates (computer_id, display_name, name, recommended, restart_required, version) values (?, ?, ?, ?, ?, ?)
Query is:
insert into available_software_updates (computer_id, display_name, name, recommended, restart_required, version) values (?, ?, ?, ?, ?, ?)
2018-02-15 15:20:10,585 [ERROR] [Tomcat-275 ] [lientCommunicationServlet] - Error processing communication content - org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; nested exception is org.hibernate.exception.DataException: could not execute statement
2018-02-15 15:20:58,262 [INFO ] [duledPool-5] [VppLicenseMonitor        ] - Running license monitor
2018-02-15 15:21:03,606 [WARN ] [Tomcat-320 ] [SqlExceptionHelper       ] - SQL Error: 1406, SQLState: 22001
2018-02-15 15:21:03,606 [ERROR] [Tomcat-320 ] [SqlExceptionHelper       ] - Data too long for column 'version' at row 1
Query is: insert into available_software_updates (computer_id, display_name, name, recommended, restart_required, version) values (?, ?, ?, ?, ?, ?

[ChristopherGlov]

That error usually means you need to truncate some data.

[ThijsX]

@beastie Thanks, i will check this out to recheck the checksum of the uploaded packages, or by removing them to check if one of them is the bottleneck!

[bvrooman]

The fact that the error mentions Data too long for column 'version' and Query is: insert into available_software_updates makes me think that you might be seeing the issue described here:
https://www.jamf.com/jamf-nation/discussions/26918/recon-an-unknown-error-occurred

The workaround listed therein resolved that same error message for me (and allowed my machines to resume inventory updates).

[ThijsX]

So, what only helped was reverting a snapshot of the server, then applied again the patch policies and it started failing again, keep restarting the MySQL service and disabled the policies, crashed Restart MySQL Service till all disabled (5) of them and all fine.

Created a support case.

Cheers,
Thijs

[ThijsX]

Seems to be product issue probs in Patch Management.

[lashomb]

Could be some schema issues if 'version' is getting a longer than expected value.

[zinkotheclown]

I believe Patch Management was introduced in Casper 9.98? I enabled it right away on all the apps available to us.

CPU and memory usage spiked during times when computer usage was high. I contacted JAMF support and after a few days, my TAM suggested I disable Patch Management. Everything went back to normal.

I have not enabled Patch Management since and I currently use AutoPkgr to inform me of app updates.