24 hours ago, I changed the location of my Kolibri Channels to an external SSD drive (SanDisk Extreme, NTFS formatted) and imported a number of large Channels (e.g. Khan Academy) into my Kolibri instance. Now, when attempting to view certain types of content (primarily video) in a Kolibri channel, I receive the following error:
“The media could not be loaded, either because the server or network failed or because the format is not supported”
In addition, the “preview images” associated with many of the “tiles” within the channel are not displayed.
There are certain channels which load and display content as expected (albeit smaller channels with non-video content), which gives me some confidence that the issue is not simply related to a failure during the process of changing the location of the Content Library.
In addition, I know that it is not an issue with the content type not being supported in my browser, as, prior to changing the location of the Content Library (and intermittently even after changing the location) I was able to view the video content successfully.
In various sections of the documentation, I saw reference to the fact that moving the location of the Content and incorporating large Channels into Kolibri can be a time consuming process.
Our team is now 72 hours away from departing for our project location, and I am at a point where I don’t know how to determine which of the following is true:
Everything is working as it should, it just requires more time to complete (however, I’ve been monitoring the Kolibri process in Task Manager on the Server device, and it is consistently at 2%)
Have we simply been too ambitious with the number and size of channels that we would like to deploy (Khan Academy, MIT Blossoms, and Career Girls being the 3 largest) and we need to eliminate content in order to address this issue?
Is there in fact a problem of some kind with my configuration that I failed to notice prior to now that is causing this behavior?
Did I simply make a poor choice with respect to the SSD we selected, and we should have gone with a different storage solution which would alleviate this problem?
Any assistance is greatly appreciated.
Thanks,
Erin
Technical details
Kolibri 0.14.7
Raspbian GNU/Linux 10 (buster) on Raspberry Pi 400 (from Raspberry Pi Imager)
Chromium Version 92.0.4515.98 (Official Build) Built on Raspbian , running on Raspbian 10 (32-bit)
Hello again @ESP ,
If the importing of the channels has finished there is no reason for the images not to appear.
There are some things I’d look at:
Run sudo dmesg and look for messages indicating some kind of disk read errors
In case you’re using nginx with kolibri (it would be good to know what kind of setup did you use to install kolibri) please do check nginx has permissions to access to the folders where the content is stored.
In case there were errors importing the channels you can try to execute kolibri manage scanforcontent so the database is updated with all the available content.
(P.S. I have done something similar to your setup in the past, and currently I’m running a Raspberry Pi 4 Model B that boots from a SSD disk without issues, but using ext4 instead of ntfs as the drive format. ext4 is a native linux system while several layers of services are needed to access to ntfs. Using standard HDs I have seen issues with ntfs in the past when many small files are read. If nothing else works, as a last desperate action I’d use an intermediate disk to copy all your content, format yours to ext4 and copy back the content to your external disk)
Thank you again for all of your continued assistance.
After reviewing your suggestions from earlier today and not finding any obvious resolutions, I built a new environment from scratch, on different (but identical) hardware, following the exact same process that I followed previously.
In this new environment, into which I loaded only two relatively small channels, I continue to experience the same issue once I complete the kolibri manage content movedirectory process; once I reverse the process and move the content back into it’s original location on the Raspberry Pi’s SD card, everything appears to work correctly.
Per your suggestion, I have run sudo dmesg on several occasions, and I see nothing indicating a disk read error.
I am indeed using Nginx - I followed the instructions included in Manual setup of a Raspberry Pi — Kolibri User Guide each time I have created a new environment (each environment being created from a clean OS image) and I have not deviated from the installation instructions, nor have I received any indication that there were any issues during the installation process. No other applications have been installed and, as you recommended, I formatted the drive using ext4 this time. With respect to permissions, I have mounted the external drive using:
sudo mount /dev/sda1 /mnt/Storage -o uid=pi,gid=pi
sudo chown -R pi /mnt/Storage
And touch /mnt/Storage/test leads me to believe that I am able to successfully access the resulting mount without any issues.
Aside from what is described above (including making no changes to the “standard” installation instructions), I have not been able to determine definitively whether or not Nginx has permissions to access the external drive and folders where the content is stored - however given that SOME content is successfully served even after the movedirectory process, I don’t necessarily have any reason to think this the issue - however, I am more than happy to try and confirm one way or another, if someone can provide me direction as to how to prove/disprove this.
I have run the kolibri manage scanforcontent process on both the original environment and the new environment, and in neither instance did I get any information indicating there was an issue, nor was the problem resolved.
As I mentioned, I also followed your suggestion and formatted the drive as ext4 - although still booting from and running on the SD card.
Ideally we’ll be able to resolve this issue so we can utilize external storage, otherwise we’ll scale back our plans for this initial deployment and limit ourselves to only the content we can serve from internal SD cards.
I have previously seen precisely these issues with serving from Nginx, so it’s important that whichever user nginx is being executed as has the necessary permissions for this folder.
If this is indeed the cause, my suspicion would be that it is caching (either in the browser, or in nginx) that is responsible for some content still being served. This might explain why video is consistently not working, as the byte range requests are not being cached.
Please take a look at which user is running nginx, and what that users permissions are on the external drive.
@ESP if you continue having problems, please send us both the nginx configuration you’re using and the nginx logs.
There are not errors in the kolibri logs you attached before, so it seems all the requests that reach kolibri are being correctly fulfiled.
If you moved the content to a different folder it might happen you need to change your nginx configuration in case you configured static folders.
One easy test would be disabling nginx and check everything works fine with just kolibri. Using nginx will provide you better performance but it can be tricky to do a correct nginx configuration.
Thank you for your suggestion regarding file permissions - it seems as if that may in fact have been at least part of my issue. While I recognize that it is far from a best practice from a security perspective, and there are certainly more elegant and/or precise ways to address this issue, I used the following command:
sudo chmod -R a+rwx /mnt/Storage
Which, I believe, ensures that any/all users have full access to the Mount associated with the SSD.
After making this change, the overwhelming majority of the content contained within Channels is displaying and playing successfully. While there are still a few channels in which video is not loading, I am no longer receiving the “The media could not be loaded, either because the server or network failed or because the format is not supported” error message - the progress indicator simply continues spinning endlessly without ever launching video. I’m still in the process of going through all of the Channels that we’ve selected, to see if there are any other issues, or to perhaps identify some commonality in the content that still fail to launch.
In answer to your question regarding the user running nginx (based on the nginx.conf file), I believe that the answer is user www-data
Thanks to you and @jredrejofor your continued support - hopefully it won’t be necessary to continue down the path of making changes to nginx.
(And while I don’t have any reason to think that either or these issues is in any way related, I’ve observed two other issues that I haven’t observed prior to yesterday in any of the testing I’ve done - the process of gathering channel listings and the importation of channel content is frequently hanging, which I’ve never observed before. In addition, as part of testing the Channels that I’ve downloaded, I noticed that parts of the MIT Blossoms and Espresso English channels were “interleafed” with one another - in other words, within the Espresso English channel there are references to MIT Blossoms content, and vice-versa. I have deleted both channels, and will attempt load them again. I only mention this on the chance that it provides you with other thoughts as to other possible issues that I may also need to address.)