Odd issue with upload using SFTPClient

[tcaduto12068]

Hi,
I am having a issue with the sftp client.

It's connecting to globalscape SFTP server and it connects fine and the upload starts fine and if I logon to the account with Filezilla the file is there in the time I expect, however the client hangs and gives a connect timeout error even thought the file was fully uploaded. If I increase the SFTP client timeout to 60 it eventually returns but with no error messages etc.

When I upload a file with another client connected to same server and account (filezilla, Paramiko python lib) I don't see this long pause after a upload.

My version of securebridge is from last december so it is almost a year old.

[tcaduto12068]

I found the issue, but I am not sure why it is an issue.

In the openAndUploadFile function in ScSFTClient on the very last line there is a final Internal upload with a comment of //CloseHandle.

If I comment out this last line I don't have to set the timeout to 60 seconds it writes the file to the server super fast.

What is the purpose of this last line which is identical to the line in the while loop? should it have _opClosingHandle instead of _opWritingFile?

I changed the last line from this:
InternalUpload(FileItem.Handle, FileItem.UploadingFileInfo, _opWritingFile);

to this:

InternalUpload(FileItem.Handle, FileItem.UploadingFileInfo, _opClosingHandle);

and this seems to fix the issue.

Can you guys verify if this is a bug or not?

[tcaduto12068]

I am guessing the last line without the _opClosingHandle was causing the server/client to wait for data that never came since the file was already uploaded.

I am guessing the purpose of that last line is to close the local file handle.

I recently renewed my subscription and will get the latest version and see if this has been fixed or not.

Would still like a confirmation if my thinking on the _opClosingHandle is correct.

Thanks :-)

[tcaduto12068]

line 1686 in in the latest scsftpclient.pas has the same issue it seems.

Code: Select all

    while not FileItem.UploadingFileInfo.EOF do
      InternalUpload(FileItem.Handle, FileItem.UploadingFileInfo, _opWritingFile);
    InternalUpload(FileItem.Handle, FileItem.UploadingFileInfo, _opWritingFile); // CloseHandle  

[ViktorV]

Unfortunately, we could not reproduce the issue.
Please check whether the problem occurs on the latest version SecureBridge 8.2.4 (you can use SecureBridge Trial Edition) and if yes, in order for us to be able to give you a detailed answer, please provide us with test access to your server.

[tcaduto12068]

I have the source code edition.
So your saying the line after the while loop should be another data write and not a close handle?

The comment right after it says CloseHandle but has a flag of opWritingFile

InternalUpload(FileItem.Handle, FileItem.UploadingFileInfo, _opWritingFile); // CloseHandle

[tcaduto12068]

Ok, I renewed my subscription for securebridge and installed the latest source code edition in Lazarus 1.8.4

And...
same issue with the timeout error while uploading a file.

I am convinced it's a bug starting at line 1684 in scsftpclient.pas

The while loop uploads the file and writes all the data until EOF, the next line I am guessing is supposed to close the file not write more data hence why that line has a comment //CloseHandle but the flag for the InternalUpload function is set to:
_opWritingFile to me that means send more data, problem is if no data is available it times out.

The line directly above the while loop is :
InternalUpload(FileItem.Handle, FileItem.UploadingFileInfo, _opOpeningFile);

And should not the line after the while loop have the flag _opClosingHandle ???

Can you guys explain what that line is supposed to do after the while loop? I can't tell what it's supposed to do based on just the flag _opWritingFile is it writing a confirm message to the server to verify the bytes received are the same as sent?
What I know is if I change that flag to _opClosingHandle it uploads the file and does not timeout on that line anymore.
without changing the flag I have to have the timeout set to 60 in order for the program to work.

The file gets fully uploaded and then hangs on that last internalupload call which is not part of the while loop unless the flag is changed to _opClosingHandle .


I can't give you guys access to the server as it's a third party that uses Globalscape managed SFTP.

[tcaduto12068]

Ok, I dug deeper into the code and the issue is actually in the
TScSFTPClient.CloseHandle function

The client fully writes the file and then that line after the while loop is doing the closehandle call if you set the flag to closinghandle it doe nothing, same as commenting out the line.

It does:
FPacketWriter.WriteAsString(Handle);
on line 1869 and then hangs in the ProcessResult on this line:

if FSSHSubSystem.ReadBuffer(TmpBuf, 0, 4) < 4 then

So whatever closehandle is sending the server does not know what to do with it and the operation times out.

i.e.

FPacketWriter.WriteAsString(Handle);


what exactly is Handle? It's being sent as a string to the server and Handle is defined as tbytes.

[ViktorV]

To solve the problem, please try to set the ScSFTPClient.ReadBlockSize and ScSFTPClient.WriteBlockSize properties first to 8192, and then to to 16384 and then to to 32768.

[tcaduto12068]

Hi,
I tried to set the block size to 4096 but it didn't have any affect.
I also connected to a standard SSH/SFTP server on Ubuntu Linux with same app and did not have this issue.
The server in question is from Globalscape. I tried filezilla and other SFTP clients and they don't have the issue connecting to the globalscape server.
For example the Paramiko SFTP client for Python works perfectly and I looked at it's code and it does not appear to be sending a remote closehandle command.

So the file uploads fine and then hangs on the closehandle. If I comment out that line it works perfectly except it does not close the local file I just uploaded.

Maybe the issue is Sbridge is doing the remote and local closehandle in the same operation?

I can try and get a hold of the server admin and see if they can look in the logs around the time I uploaded the files and see if anything shows up. the problem is it's a third party and who knows if they will bother to look at it.

[tcaduto12068]

well, this is weird, I set the sftpclient read/write blocksize to the same size as the socket read/write buffer in the sshclient component which is 32768 and the timeout issue seems to be gone.
The SFTP client block size defaulted to 65536 which is double what the socket buffer was.

So do you guys know what the deal with this is? Should I set them the same for all my apps that use the SFTP client?

[tcaduto12068]

It's still taking a really long time to upload a tiny 250 byte file.
It's doing it before the 15 sec timeout but it should be much faster. I switched everything back to the default sizes and it's still working so not sure what is going on.
the Python Paramiko sftpclient on same machine to same server is MUCH MUCH faster which seems to be the result of it not doing a remote closehandle.

[tcaduto12068]

it started doing it again, so I did this and now I can upload the file and skip the remote close and close the file stream so the local file is not in use:
while not FileItem.UploadingFileInfo.EOF do
InternalUpload(FileItem.Handle, FileItem.UploadingFileInfo, _opWritingFile);
if fileitem.UploadingFileInfo <> nil then
FileItem.UploadingFileInfo.TryCloseStream;
// InternalUpload(FileItem.Handle, FileItem.UploadingFileInfo, _opWritingFile); // CloseHandle

Is doing a remote closehandle something new in the SFTP protocol?

[tcaduto12068]

I also noticed if the upload times out because of the close handle it jumps to the exception handler and times out again from this line:

except
on EScSFTPError do begin
if FFileList.IndexOf(FileItem) >= 0 then
CloseHandle(FileItem.Handle); <---- times out again and leave local file open
end;
end;

Would it not be better to do this instead so the local file gets closed:
if fileitem.UploadingFileInfo <> nil then
FileItem.UploadingFileInfo.TryCloseStream;

[ViktorV]

Calling the CloseHandle method should close the file on the server. In case you will not call the CloseHandle method, the file will not be closed and the problems may appear. We cannot reproduce the case of hanging in our environment (we use the EFS server from GlobalScape for testing). Perhaps the reason for such behavior is related to your environment (customization of your SSH server, network infrastructure, network settings and restrictions, etc.). Therefore, please, try to provide us with test access to your server for problem investigation.
Also, please, set the TScSFTPClient.PipelineLength property to 1 and test again.