Upload progress bar

[manaioussema]

Hi, first thank you for this awesome library.
I have a project that requires sometime uploading large files, and the user complained about the absence of a hint or a message during upload. I managed to make other threads during the normal button click but I couldn't find how to do it with the upload button click.
Any advice towards the solution is much appreciated.

[johnperry-math]

Could this be addressed by adding a progress listener to the XMLHttpRequest created in Remi.prototype.uploadFile, and a corresponding onprogress event listener to remi's FileUploader? (Looking for example at this stackoverflow answer.)

[dddomodossola]

I am doing some tests, this is the javascript I created:

                Remi.prototype.uploadFile = function(widgetID, eventSuccess, eventFail, eventData, file){
                    var url = '/';
                    var xhr = new XMLHttpRequest();
                    xhr.onprogress = function(event){
                        if(event.lengthComputable){
                            var params={};
                            params['file_name'] = 'filename'/* file.name*/;
                            params['loaded'] = event.loaded;
                            params['file_size'] = event.total;
                            
                            remi.sendCallbackParam('%(emitter_identifier)s','onprogress',params);
                        }
                    };

                    var fd = new FormData();
                    xhr.open('POST', url, true);
                    xhr.setRequestHeader('filename', file.name);
                    xhr.setRequestHeader('listener', widgetID);
                    xhr.setRequestHeader('listener_function', eventData);
                    xhr.onreadystatechange = function() {
                        if (xhr.readyState == 4 && xhr.status == 200) {
                            /* Every thing ok, file uploaded */
                            var params={};params['filename']=file.name;
                            remi.sendCallbackParam(widgetID, eventSuccess,params);
                            console.log('upload success: ' + file.name);
                        }else if(xhr.status == 400){
                            var params={};params['filename']=file.name;
                            remi.sendCallbackParam(widgetID,eventFail,params);
                            console.log('upload failed: ' + file.name);
                        }
                    };

                    fd.append('upload_file', file);
                    xhr.send(fd);
                };

and the python code:

import remi.gui as gui
from remi import start, App
import os


class FileUploaderWithProgressEvent(gui.FileUploader):
    @gui.decorate_event
    def onprogress(self, file_name, loaded, file_size):
        return (file_name, loaded, file_size)


class MyApp(App):

    def main(self):
        # creating a container VBox type, vertical (you can use also HBox or Widget)
        main_container = gui.VBox(width=300, height=200, style={'margin': '0px auto'})

        self.progress = gui.Progress(0,100)
        main_container.append(self.progress)

        self.bt_start_upload = gui.Button("Start upload")
        

        self.file_uploader = FileUploaderWithProgressEvent("C:\\Users\\progr\\Desktop\\remi\\examples\\")
        self.file_uploader.onprogress.do(self.onprogress_listener)
        self.file_uploader.onsuccess.do(self.fileupload_on_success)
        main_container.append(self.file_uploader)

        # returning the root widget
        return main_container
    
    def onprogress_listener(self, emitter, file_name, loaded, file_size):
        print(file_name, loaded, file_size)

    def fileupload_on_success(self, emitter, filename):
        print('File upload success: ' + filename)


if __name__ == "__main__":
    # starts the webserver
    start(MyApp, address='0.0.0.0', port=0, start_browser=True, username=None, password=None)

Today I was unable to make javascript to trigger the onprogress event. I will do some further tests shortly, and commit on a specific branch.

[johnperry-math]

I'm able to make javascript trigger the onprogress event by modifying your code from this:

xhr.onprogress = function(event){
    // ... snip
}

to this:

xhr.upload.addEventListener('progress', function(e) {
    console.log('progress!');
    // ... snip
});

I don't understand why that should matter, since several examples I find on the set it up the same way as yours, but I do get the progress! message in the console. I don't have the callback working quite yet.

I should probably add that I also added a few lines to make sure my local remi repo is used instead of the pip-installed version. That threw me for a loop a bit, because I was also checking for onloadstart and that console message wasn't showing up, either.

[johnperry-math]

I don't have the callback working quite yet.

...and that's solved by changing this line:

remi.sendCallbackParam('%(emitter_identifier)s','onprogress',params);

...to this:

remi.sendCallbackParam(widgetID,'onprogress',params);

Altogether, the Javascript looks like this:

                    xhr.upload.addEventListener('progress', function(e) {
                        console.log('progress!', widgetID, %(emitter_identifier)s, e.loaded, e.total);
                        if(event.lengthComputable){
                            var params={};
                            params['file_name'] = 'filename'/* file.name*/;
                            params['loaded'] = event.loaded;
                            params['file_size'] = event.total;
                            console.log("length is computable; sending callback");
                            remi.sendCallbackParam(widgetID,'onprogress',params);
                        }
                    });

I didn't modify the Python at all, except (a) to inject the local remi repository and (b) to modify the path from C:\\Users\\progr\\Desktop\\remi\\examples\\ to something appropriate for my machine.

Let me know if that works for you 😁 🎉 or not 😭

[dddomodossola]

@johnperry-math thank you a lot.
I made the suggested changes and commited to master branch.
You can test it with the example "progress_bar_app.py".
Please give it a try, if it is ok we can close this issue. ;-)

[johnperry-math]

Looks to me as if it works fine! I uploaded a large file, watched the progress bar update, then checked the console and saw relevant "progress" messages. I think the issue can be closed.

Out of curiosity, I do see these messages in the console from time to time, on multiple machines, including in this example:

An attempt was made to use an object that is not, or is no longer, usable

Do you know what causes that, whether it's of concern, and/or how to avoid it?

[manaioussema]

Thank you for the solution.
What is the minimum file size in order to see the progress bar? I tried the example with 20Mo file and I saw no progress, just went from empty to full.

[johnperry-math]

@manaioussema I've managed to see the progress bar update with a roughly 10Mb file, certainly with 17Mb. You can try a larger file, but if your machine has blazingly fast I/O you could also check the web developer console. I saw the following in mine during & after the 10Mb upload:

opening websocket [127.0.0.1:42205:48:33](http://127.0.0.1:42205/)
progress! 139664292703760 139664292621200 1670532 10586417 [127.0.0.1:42205:231:33](http://127.0.0.1:42205/)
length is computable; sending callback [127.0.0.1:42205:237:37](http://127.0.0.1:42205/)
progress! 139664292703760 139664292621200 4586884 10586417 [127.0.0.1:42205:231:33](http://127.0.0.1:42205/)
length is computable; sending callback [127.0.0.1:42205:237:37](http://127.0.0.1:42205/)
progress! 139664292703760 139664292621200 10586417 10586417 [127.0.0.1:42205:231:33](http://127.0.0.1:42205/)
length is computable; sending callback [127.0.0.1:42205:237:37](http://127.0.0.1:42205/)
upload success: ErtecWest1983_hazard-aleutian-shelf.pdf [127.0.0.1:42205:252:37](http://127.0.0.1:42205/)
An attempt was made to use an object that is not, or is no longer, usable 2

If you don't see something like that and you have remi installed via pip, make sure you're not inadvertently running the pip version. I have remi installed via pip, so When I run examples and I want to use the latest version of remi in the folder, I add this to the beginning of the example:

import sys
sys.path.insert(0, "/path/to/remi")

...where of course I replace /path/to by the path to remi.

Hope that helps.