I have NGINX server installed and running with other apps (just migrated from Apache). Kolibri-server does not load/run with NGINX, and disables NGINX.
Technical details
Kolibri version: Kolibri-server V 0.3.6-0ubuntu1
Operating system: Ubuntu 18.04
Browser: Firefox 85.0.1
Detailed information: Before installing Kolibri-server NGINX is running and serving other apps. After Kolibri-server install NGINX fails due to a missing NGINX conf.d Kolibri file pointer missing. Kolibiri-server is loaded, but not active. I am not able to start Kolibri setup in the browser on default port 8080 (or any other port). If I remove the Kolibri file in Nginx conf.d Nginx starts again cleanly, but Kolibri still will not load. I’ve removed and reinstalled everything a couple of times, but without any luck. How can I get this enabled? It ran fine with Apache, but I had many reasons to move to Nginx.
I’m attaching the following files (in zipped folder):
Kolibri log file before conf.d file removed
Kolibri log after conf.d file removed
Journalctl -xe results following attempted startup of Kolibri and Nginx
Screenshot of error message when attempting to open Kolibri file in Nginx conf.d
Update #1: I removed kolibri-server and all files/folders, and installed kolibri 0.14.6 (base package, not server version). That works fine. The issue appears to be limited to the server package only.
Update #2: I tried installing kolibri-server on top of kolibri base. There were 2 errors: one was related to an include statement which I removed (did not appear to be needed). The other is due to a missing uwsgi.ini file in the /etc folder. Is it possible to get a copy of that file so I can manually install it?
FWIW: I think this problem is a result of a re-install after I partitioned my device to add more storage. It appears that there were broken links left from the previous install when I removed kolibri. I would appreciate any help getting it resolved.
Hi @markknit , I’m sorry but I’m a bit confused with your email.
First of all, the logs you attached have their lines cut, for example, this line would be important to know what’s hapening with your system: Feb 14 16:37:01 ares-server kolibri-server[3755]: realpath() of /etc/kolibri/dist/uwsgi.ini failed: No s
Also, could you add a bit of more detail to your explianation, when you said there’s an include you removed, what include is it? And when you say a missing uwsgi.ini file, kolibri-server does have an uwsgi.ini file that’s installed in /etc/kolibri/dist if you don’t have it it’s quite clear your packaging system is having some kind of problems. Probably because you removed manually some files that now are not being installed because they are conf files. If that the case, please do sudo dpkg --purge kolibri-server
and then install it again normally
Appart from that, your kolibri database is corrupted, at least the one to execute the tasks, in the logs you sent it is raising errors with or without kolibri-server. I see problems in the database that handles the background tasks. If you don’t have anything important it it I’d delete it (files job_storage.* in your /home/ares/.kolibri/ folder)
I can’t tell much more due to the half-cutted logs but I hope this helps.
Regards
José
I followed your instructions about removing kolibri-server (and kolibri), followed by ‘sudo dpkg --purge kolibri-server’, followed by ‘sudo apt autoremove’. I also deleted the .kolibri folder in my home directory, and deleted the Kolibri folder in /etc. When that was done I rebooted and did a normal install. This time the installer ran normally (with all of the expected prompts), and when completed ‘sudo systemctl status kolibri-server’ shows that it is loaded and active. So good news!!
Unfortunately I am encountering a different problem now: when I open my browser and point it to: ‘127.0.0.1:8080’ kolibri is not loading. It actually did start after the initial re-install, so I ran through the normal setup routines. However, at the very end of the process it said there was an error (no codes, etc.), and then it stopped loading – even though the service is still active. I went through the entire remove/purge/install process again after that, but now the setup routine will not load at all – there is no response when I load the default kolibri page. (BTW: the firewall is turned off). I ran ‘lsof -i :8080’ and it comes up empty – i.e. kolibri is not listening on that port (nothing else was either). I also checked ports 80 & 8000 – it’s not listening there either. I also checked nginx – it’s loaded and active. Same result with Netstat.
I’m hoping you might have some insight as to what is preventing kolibri from activating properly. I’m attaching 2 log files from the system. I can gather any other information you need – just let me know.
hi @markknit , all the information in the attached logs say everything is fine, kolibri started correctly in your system without any problem. These are kolibri logs, so I suppose the problem must be in the nginx part, please attach here the logs you have at /var/log/nginx
You say nginx is active after uninstalling kolibri, that’s also fine as kolibri-server tries to be non intrusive in case there are more services running nginx, so it does not uninstall nor modify its current configuration.
I have some questions that should help troubleshooting your case:
At the beginning of the thread you say you moved from apache, could it be there are some apache processes still running?
Also, can you tell me what port are you chosing when installing kolibri-server and what kind of machine are you using? I have seen some delays before kolibri is completely usable in some low-resources machines as Raspberry Pi. It starts but, specially the first start, can take some minutes in these cases.
Last but not least: if you’re chosing port 80 when installing kolibri-server, don’t do it. In Ubuntu and Debian they have the default nginx configuration to use port 80. As kolibri-server respects previous configurations it can’t use it. It’s a known issue that hopefully will be soon fixed but in the meantime better choose 8080 or 8008.
Apache: I removed and purged Apache when I installed Nginx, so nothing from Apache could be running now.
Ports: I have had the port set at :8080 all along. I did try another random port (8990) to see if that would help, but it didn’t so I switched back to 8080. Related to that: I checked port 8080 and it does not show any service listening on it. I also did a full listing of all ports to see if Kolibri showed up somewhere else - it does not appear to be listening on any port.
Server: this is an Intel Celeron N3360 quad core server designed to handle around 50 users simultaneously. It has 8GB of Ram and uses SSD. In general it runs all applications and services very quickly.
I have another idea that might help with troubleshooting: I removed both kolibri-server and kolibri, and then re-installed kolibri (not the server). It works! I can see it listening on port 8080.I then installed kolibri-server on top of kolibri (base), and it stopped working, and is no longer listening on port 8080 (or any port). I removed kolibri-server and started kolibri base and it works again. So, the problem appears to be specific to kolibri-server. Does that help? Normally I would be content to just use the base server, but I’m installing a set of new systems in some very large schools in Kenya, so the extra performance of kolibri-server could help a lot.
I’m attaching the kolibri logs that show kolibri listening on :8080, but not kolibri-server.
Jose: Good news (mostly)! I believe I found a (partial) solution to the problem of Kolibri-server not running, and more specifically not listening on port 8080. I compared every aspect of the kolibri base installation with the kolibri-server installation and noticed that in the latter Cherrypy_start is set to false in the Options.ini file. However, with the Kolibri base installation it is turned on, and it works. So, I changed the setting to cherrypy_start = true, rebooted, and now the kolibri -server is listening on port 8080, and the system works normally. For some reason every time I download/install Kolibri-server that setting is being set to false. I suspect that is happening at some point after the install, as I noticed that afterwards I did a stop/start of the server and found that it was not working again. I checked the options.ini file and found that cherrypy_start had been reset to false again. When I put it back to true and restarted the server operation went back to normal.
So… I have a couple of questions:
Do you have ANY idea what is causing the Cherrypy_start parameter to be reset periodically (or at install)?
Should I remove and reinstall Cherrypy? If so, is there a specific release/tool that I should use to do that?
It helps to know what the source of the problem is, but I need to make sure that it won’t get turned off during operation, as that would cause a failure after a restart and I will not have any technicians around at the schools that can do that.
hi @markknit I’m afraid your good news are not correct.
What you have seen is the right behaviour when installing kolibri-server.
When you install kolibri alone, it starts a web server plus a queue to handle async events. As any Python application it only can run one thread at a time so it only can use one core of the four your server has. In servers needing having many concurrent users this can be a problem, and that’s the reason to provide the kolibri-server package. If you don’t have many concurrent users you can live with kolibri alone.
When you install kolibri-server it adds the parameter you have seen to avoid kolibri starting the web server: The web server is handled by nginx that uses kolibri to response to the requests and you use all the cores your hardware has. There is only one remaining process running kolibri to manage the async queue, but not web requests.
So, if you want to use kolibri-server, with nginx, don’t change manually the parameter you’re changing. In any case, you have checked it’s useless because kolibri-server will restore it in every restart.
It’s clear your problem is in nginx, but I could not download the logs because the dropbox links you provided in a previous message have been removed.
OK, thanks for setting me straight Jose. I understand the situation now. I’m reattaching the Dropbox links with the Nginx logs and the config files in Kolibri and Nginx. I don’t think the log files are going to help. They show activity related to other sites accessed, but not for attempting to access Kolibri on port 8080.
Is it possible for you to send me a default/standard Nginx.conf file for Kolibri, and the Kolibri config file in the Nginx Sites Available folder (which is symlinked to Sites Enabled)? If so I can pug those in to see if that helps.
hi @markknit I have taken a look at your logs and conf files and I only can tell that:
nginx logs are empty because your nginx configuration is not correct
your nginx configuration in /etc/kolibri is incorrect and it’s not the configuration kolibri-server writes , it has been clearly modified manually adding things that are making it unusable (if you want to see the differences, just compare it with kolibri-server/nginx.conf at master · learningequality/kolibri-server · GitHub )
Please, if you want to use kolibri-server as it’s intended:
apt-get purge kolibri
delete all the files in /etc/kolibri
apt-get install kolibri-server
will do a clean installation and it should work (don’t choose the port 80 for kolibri as in your nginx logs I see you have nginx configured to run other services in port 80)
Thanks Jose. I had realized that Nginx was the source of the problem. I did try some different config files (as you noticed) but the real problem was that Nginx was not functioning due to unresolved dependencies. I finally decided to do a clean Ubuntu install and then added Nginx and Kolibri back in. It’s all working now. Thanks for your help.