# Automating and Updating Large Channels in Kolibri Studio
**URL:** https://community.learningequality.org/t/automating-and-updating-large-channels-in-kolibri-studio/3342
**Category:** Support
**Tags:** content
**Created:** [September 19, 2024, 6:22am UTC](https://community.learningequality.org/t/automating-and-updating-large-channels-in-kolibri-studio/3342 "2024-09-19T06:22:04Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@essa](https://community.learningequality.org/u/essa)
#### Post date: [September 19, 2024, 6:22am UTC](https://community.learningequality.org/t/automating-and-updating-large-channels-in-kolibri-studio/3342/1 "2024-09-19T06:22:04Z")
Hello,
I’m working for an organization and need to upload content from our [website](https://psl.org.pk/) to Kolibri Studio.
I’m making a channel called **PSL (Pakistan Sign Language) Dictionary**. I have around **80 categories** and an average of **100 videos per category**.
What would be the best (efficient) way to **automate** this process keeping crucially in mind that **adding/removing videos** (and its metadata) as well as **adding/removing categories** (and its metadata) would be inevitable in the future.
From my experience, there are potentially 2 options (both involve **ricecooker** ):
1. Writing a **sushi chef script** which reads in data (csv) and creates the channel and its contents. For updating, we would implement conditioning in the csv and subsequent conditioning implementation in the script which would execute by **re-running** the script. For example; to remove a video, specifying a “delete” column in the csv and setting it 1, and handle it accordingly in the script when re-running. This re-running mechanism seems inefficient since ricecooker downloads and uploads files even when they are in stored in the cache. Especially considering the case if i have a huge channel and i simply want to edit a video’s name (not possible manually since API generated channels not editable), then my script would re-run processing 1000s of videos all over again.
2. Using the [CSV Workflow](https://github.com/learningequality/sample-channels/tree/master/channels/csv_channel) which automates the whole uploading process given that the channel and its contents follow a specific csv format. This has a huge limitation in terms of updating content.
In conclusion, the main issue seems to be in **updating** content once its uploaded on the studio. Given that the **corrections workflow** and the `StudioApi` doesn’t work anymore as mentioned in:
> [@Corrections workflow - Ricecooker](https://community.learningequality.org/t/corrections-workflow-ricecooker/3336/4):
>
> Sorry, the documentation is not up to date. The corrections workflow is not supported, and will not work. I will update the documentation, thanks for bringing this to our attention.
We are left with a big hole in terms of updating the content once uploaded. Hence, let me know your thoughts and the main query of what is the best approach to tackle this, given ricecooker’s limitations? Thank you.
Regards,
Essa, [FESF](https://fesf.org.pk/)
---
### Author:  [@jredrejo](https://community.learningequality.org/u/jredrejo)
#### Post date: [September 19, 2024, 3:25pm UTC](https://community.learningequality.org/t/automating-and-updating-large-channels-in-kolibri-studio/3342/2 "2024-09-19T15:25:21Z")
Hello @essa
First, if you’re using `ricecooker`, you can update the channel in the future by using the same tool. There’s no need to use the deprecated _corrections_ workflow. Simply use `ricecooker` again with the same channel ID and your API token to make updates.
Regarding the option to upload the channel, it’s a tradeoff:
- **CSV Workflow** : Requires less effort compared to writing a chef script but offers less flexibility in how the channel is structured.
- **Chef Script** : Provides more flexibility but will require additional work to set up and maintain.
Ultimately, the best approach depends on your specific needs and available resources. Keep in mind, since you’re using `ricecooker`, you can switch between the two methods for different uploads if necessary.
Best regards,
José
---
### Author:  [@jredrejo](https://community.learningequality.org/u/jredrejo)
#### Post date: [September 19, 2024, 4:07pm UTC](https://community.learningequality.org/t/automating-and-updating-large-channels-in-kolibri-studio/3342/3 "2024-09-19T16:07:41Z")
Hello again @essa
I forgot to comment on this from your first message:
> [@essa](#):
>
> This re-running mechanism seems inefficient since ricecooker downloads and uploads files even when they are in stored in the cache
This should not be happenning unless you’re deleting ricecooker cache in your _local_ disk. If you do it, videos will be redownloaded and processed again. If this is happening to you it’s a bug we are not aware and should be fixed.
Regards
José
---
### Author:  [@essa](https://community.learningequality.org/u/essa)
#### Post date: [September 24, 2024, 9:09am UTC](https://community.learningequality.org/t/automating-and-updating-large-channels-in-kolibri-studio/3342/4 "2024-09-24T09:09:07Z")
Hello José,
I did try the CSV Workflow approach and did verify your following point through seeing the send/receive speed under the Performance tab in my Task Manager:
> [@jredrejo](#):
>
> This should not be happenning unless you’re deleting ricecooker cache in your _local_ disk.
I think the reason I had this confusion was due to the fact that when you use either method (sushichef or csv workflow) and given that you already have files in the ricecooker cache, the console still prints out a few statements that may seem ricecooker is re-downloading and re-uploading everything again.
Like for example the following log entry says that it found the particular video from the ricecooker storage, but then ricecooker is supposedly **uploading** the file **again** on Kolibri Studio (?):
```auto
INFO 2024-09-17 16:23:02 VideoResource - Video preset from /home/essa_fesf/ricecooker-psl-content/ricecooker-script/storage/f/4/f41a1f90ef808dd8211c8d14a281f68f.mp4 = high resolution
INFO 2024-09-17 16:23:03 root - Uploaded cfd34a1b335e458e451d256ddd7eed5c.mp4 (162/1183)
```
Another query/confusion is related to ricecooker creating the tree all over again whenever you **re-run** a script, as indicated in the below log message:
```auto
INFO 2024-09-17 16:28:58 root - Creating channel...
INFO 2024-09-17 16:28:58 root - Creating tree on Kolibri Studio...
INFO 2024-09-17 16:28:58 root - Creating channel Adjectives
INFO 2024-09-17 16:28:59 root - Preparing fields...
INFO 2024-09-17 16:28:59 root - (0 of 593 uploaded) Processing Adjectives (ChannelNode)
```
Please let me know your clarity on this. Thank you!
Regards,
Essa, [FESF](https://fesf.org.pk/)
---
### Author:  [@jredrejo](https://community.learningequality.org/u/jredrejo)
#### Post date: [September 24, 2024, 12:17pm UTC](https://community.learningequality.org/t/automating-and-updating-large-channels-in-kolibri-studio/3342/5 "2024-09-24T12:17:34Z")
Hello @essa
About your questions,
> [@essa](#):
>
> Like for example the following log entry says that it found the particular video from the ricecooker storage, but then ricecooker is supposedly **uploading** the file **again** on Kolibri Studio (?):
no, it does not upload them if the hash of the files are the same. If you haven’t removed your local cache nor modify a file it won’t be uploaded. You see the logs telling the video was already uploaded (thus 1 second before the video is seen until you see the message of uploaded)
> [@essa](#):
>
> Another query/confusion is related to ricecooker creating the tree all over again whenever you **re-run** a script, as indicated in the below log message:
>
> Please let me know your clarity on this. Thank you!
I don’t quite understand what you mean: where’s the issue? ricecooker shows the logs of the tree and will modify it in Studio if needed. Seeing the tree in the logs does not mean it’s been changed, it means ricecooker needs to detect any difference from your new local tree and the one in the Studio server to decide if changes are needed.
Regards
José