Andrius Rinkevicius
2014-05-16 09:00:54 UTC
Olivier,
I am working at the moment for making File Upload/Download available on
MacOSX (by the way, would it be more correct to call Sync Directories or
like that?), and I need advice.
Mounting of ftp/sftp shares is not possible on OSX (gvfs/dbus are not
ported). So, we need to use other software for mounting, and the only open
source and free option available is MacFUSE/Macfusion. It works reasonably
well, however, it seems that it is essentially single threaded application.
While Upload/Download code uses multiple threads. As result, when
uploading/downloading I am getting multiple errors of "Failed copy file"
with "Permission denied". Especially when downloading from remote directory.
I already tried following:
1. Reduced number of workers in queues to 1. This improved situation.
Instead of tens of errors I was getting just few. Flat directories (without
sub-directories) was copied mostly without errors. However, if there was
deeper directory structure, there was still a lot of errors.
2. Replaced g_file_copy_async() with synchronous version. This probably
improved situation a little bit, but still there is number of errors. If
errors are produced I can push Download button several times, each time the
number of errors is smaller until finally there is zero errors and
directories are synced. The number of pushes needed correlates to the depth
of directory tree.
It looks that recursive calls to walk_local_directory() in
walk_local_directory_job() might be an issue. I am thinking how to solve
this issue. Maybe, instead of calling walk_local_directory() recursively we
can add uri to list, and walk it in idle callback?
Any suggestions are welcome.
Andrius
I am working at the moment for making File Upload/Download available on
MacOSX (by the way, would it be more correct to call Sync Directories or
like that?), and I need advice.
Mounting of ftp/sftp shares is not possible on OSX (gvfs/dbus are not
ported). So, we need to use other software for mounting, and the only open
source and free option available is MacFUSE/Macfusion. It works reasonably
well, however, it seems that it is essentially single threaded application.
While Upload/Download code uses multiple threads. As result, when
uploading/downloading I am getting multiple errors of "Failed copy file"
with "Permission denied". Especially when downloading from remote directory.
I already tried following:
1. Reduced number of workers in queues to 1. This improved situation.
Instead of tens of errors I was getting just few. Flat directories (without
sub-directories) was copied mostly without errors. However, if there was
deeper directory structure, there was still a lot of errors.
2. Replaced g_file_copy_async() with synchronous version. This probably
improved situation a little bit, but still there is number of errors. If
errors are produced I can push Download button several times, each time the
number of errors is smaller until finally there is zero errors and
directories are synced. The number of pushes needed correlates to the depth
of directory tree.
It looks that recursive calls to walk_local_directory() in
walk_local_directory_job() might be an issue. I am thinking how to solve
this issue. Maybe, instead of calling walk_local_directory() recursively we
can add uri to list, and walk it in idle callback?
Any suggestions are welcome.
Andrius