In case of SQLite database login and password is not used. There is no need to check "Create database and database user before initialization". Just fill in "Path to database file", folders in that path should exist after installer finishes extracting files, so C:\NetXMS\server.db should be fine. NetXMS will create the database automatically.
And you don't need to run installed again, you can just put this into netxmsd.conf:
DBDriver=sqlite.ddr
DBName=C:\NetXMS\server.db
And run
nxdbmgr init
this should initialize the database.
And you don't need to run installed again, you can just put this into netxmsd.conf:
DBDriver=sqlite.ddr
DBName=C:\NetXMS\server.db
And run
nxdbmgr init
this should initialize the database.