Duplicate entries for software_inventory

Started by 2b2bff, February 25, 2021, 03:07:05 PM

Previous topic - Next topic

2b2bff

Hi again,

I have a similar issue as already been written about a while ago. I have a repeating error message in log, that it refuses to write a line in software_entry because of a duplicate. However it seems like certain UTF-8 codes are replaced by "?" what makes them collide. Screenshot is attached, CSV export looks like this:

"Windows Phone 8.1 Tools for Visual Studio 2015","14.0.25527","Microsoft Corporation","19.09.2016","",""
"Windows Phone 8.1 Tools for Visual Studio 2015 - ENU","14.0.25420","Microsoft Corporation","19.09.2016","",""
"Windows Phone 8.1 Tools for Visual Studio 2015 - ???","14.0.25420","Microsoft Corporation","19.09.2016","",""
"Windows Phone 8.1 Tools for Visual Studio 2015 - ???","14.0.25420","Microsoft Corporation","19.09.2016","",""
"Windows Phone 8.1 Tools for Visual Studio 2015 � PLK","14.0.25420","Microsoft Corporation","19.09.2016","",""

So the log complains about a duplicated software that in reality isn't.

What can I do about this?

Cheers Frank


Victor Kirhenshtein

Hi,

what server version do you have and on what OS? If Linux, was it installed from packages or built from source?

Best regards,
Victor

2b2bff

Hi,

it is 3.8.166, running on Windows Server 2019, using an Microsoft SQL Server 2016 as backend...

Cheers Frank

Victor Kirhenshtein

Hi,

seems that there are two different problems. One is with CSV export - currently management console uses current system encoding instead of UTF-8, which can produce files like you have shown. I just fixed that, next version will always use UTF-8 for CSV encoding.
Second issue is likely database encoding - as we are using varchar fields text is converted to whatever character set is choosen for database, with all unsupported Unicode characters converted to ?. Easiest option probably would be to switch database to UTF-8 encoding, but is is only available for SQL Server 2019. If you only encounter such characters in software inventory, you can try to change data type for columns in table software_inventory to NVARCHAR from VARCHAR. I have registered issue in our bugtracker for considering switching to NVARCHAR for MS SQL: https://track.radensolutions.com/issue/NX-2016 - if implemented it will solve issues like this one.

Best regards,
Victor


2b2bff

#4
Sounds reasonable - thanks...

However changing the name column to nvarchar did not help. I guess you already convert the charset before sending it to the database...
I will just wait for the bugfix..