SSH Client perpetually connecting

Discussion of open issues, suggestions and bugs regarding network security and data protection solution - SecureBridge
Post Reply
ED-Clint
Posts: 36
Joined: Thu 18 May 2017 08:52

SSH Client perpetually connecting

Post by ED-Clint » Mon 12 Nov 2018 09:14

Hi again,

I have an SSH Client with which I have a log in both the BeforeConnect and AfterConnect (as well as BeforeDisconnect and AfterDisconnect) so I can see what is happening during some connection testing.

I have seen a few odd logs where there is a Connecting message (from the BeforeConnect) but no following Connected message (from the AfterConnect). I have the connection attempt in a try except so if the connection fails I also get a log of the failure as in the except I log any exception.

So I should see a Connecting then a Connected or a Connecting then a Failure. However what I see is the Connecting message and then nothing else until I manually restart the connection. It is as if the connection just hangs and never moves on. What could cause this?

Just to be clear this happens maybe once in a 72 hour period and it is very likely it is happening when there is a connection problem on the network (a link drop or similar), in fact the last one that presented this behaviour was definately during a network connection problem and when I got back on to the server I saw just the Connecting with no subsequent Failure message.

I should also add that after the Connect line in Delphi I have an If statement to check that the SSHClient.Connected is true and if not then I repeat until there is a connection. In the repeat loop I log the failure, sleep for 5 minutes, connect again. Nothing from this loop happened in over 15 minutes, so it really looks like the Connecting happened then the program completely hung at this point, with no exception (at least none logged) and I am not sure what could cause it to get in to this state.

To give you some code;

Code: Select all

with SSHClient do
  begin
    KeyStorage := keyStore;
    Options.ServerAliveCountMax := 3;
    Options.ServerAliveInterval := 18;
    Options.TCPKeepAlive := true;
    Timeout := 90;

    OnServerKeyValidate := ServerKeyValidate;
    BeforeConnect := SSHConnectorBeforeConnect;
    BeforeDisconnect := SSHConnectorBeforeDisconnect;
    AfterConnect := SSHConnectorAfterConnect;
    AfterDisconnect := SSHConnectorAfterDisconnect;
  end;

procedure SSHClientStart;
begin
  try
    with SSHClient do
    begin
      Hostname := SSHHostname;
      Port := SSHPort;
      User := SSHUsername;
      if SSHType = 0 then Authentication := atPassword;
      if SSHType = 1 then Authentication := atPublicKey;
      Password := SSHPasswd;
    end;
    with SSHChannel do
    begin
      Client := SSHClient;
      DestHost := ChannelHost;
      DestPort := ChannelDPort;
      SourcePort := ChannelSPort;
    end;
    SSHClient.Connect; // The program just seems to hang here.
    if SSHClient.Connected then
    begin
      SSHChannel.Connect;
    end
    else
    begin
      repeat
        Log('Connection Failure'); // The Log procedure writes a line to a local MySQL database table. I never see this log.
        sleep(300000);
        SSHClient.Connect;
      until (SSHClient.Connected);
      SSHChannel.Connect;
    end;
  except on E: Exception do
    Log(e.ClassName+e.Message); // I never see this log.
  end;
end;

procedure SSHConnectorBeforeConnect(Sender: TObject);
begin
  Log(' Connecting...'); // When this issue occurs I do not see multiple of this log, so the repeat until loop has not been entered.
end;

procedure SSHConnectorAfterConnect(Sender: TObject);
begin
  Log(' Connected.'); // When this issue occurs I do not see this log.
end;

ViktorV
Devart Team
Posts: 3168
Joined: Wed 30 Jul 2014 07:16

Re: SSH Client perpetually connecting

Post by ViktorV » Tue 13 Nov 2018 15:08

The AfterConnect event occurs only after a successful connection to SSH server. If an error occurs while executing the TSSHClient.Сonnect method, the error will be raised and the code won't be executed further.
Therefore, to solve the task, you should change the architecture of your application taking into account the above.

Code: Select all

repeat
  try
    SSHClient.Connect; // The program just seems to hang here.
  except
    Log('Connection Failure'); // The Log procedure writes a line to a local MySQL database table. I never see this log.
    sleep(300000);
  end
until SSHClient.Connected;
SSHChannel.Connect;

ED-Clint
Posts: 36
Joined: Thu 18 May 2017 08:52

Re: SSH Client perpetually connecting

Post by ED-Clint » Wed 14 Nov 2018 07:51

Thank you for the suggestion Viktor, I will try this. However I still do not fully understand why the initial connect did not give me the exception log;

Code: Select all

procedure SSHClientStart;
begin
  try
    // Setup connection parameters
    SSHClient.Connect; // The program just seems to hang here.
    if SSHClient.Connected then
    begin
      SSHChannel.Connect;
    end
    // the repeat loop was here
  except on E: Exception do
    Log(e.ClassName+e.Message); // I never see this log. Surely if the SSHClient.Connect failed I should see a log here?
  end;
end;

ED-Clint
Posts: 36
Joined: Thu 18 May 2017 08:52

Re: SSH Client perpetually connecting

Post by ED-Clint » Wed 14 Nov 2018 11:11

Unfortunately the same situation just happened, the SSHClient gives me the Connecting... log but then there is just nothing, no failure and no retry in the repeat loop;

Code: Select all

procedure SSHClientStart;
begin
  try
    with SSHClient do
    begin
      Hostname := SSHHostname;
      Port := SSHPort;
      User := SSHUsername;
      if SSHType = 0 then Authentication := atPassword;
      if SSHType = 1 then Authentication := atPublicKey;
      Password := SSHPasswd;
    end;
    with SSHChannel do
    begin
      Client := SSHClient;
      DestHost := ChannelHost;
      DestPort := ChannelDPort;
      SourcePort := ChannelSPort;
    end;
    repeat
      try
        SSHClient.Connect;
      except
        Log('Connection Failure');
        sleep(300000);
      end
    until (SSHClient.Connected);
    SSHChannel.Connect;
  except on E: Exception do
    begin
      Log(e.ClassName+e.Message);
  end;
end;
What could cause there to be no exception on a connection attempt?

ViktorV
Devart Team
Posts: 3168
Joined: Wed 30 Jul 2014 07:16

Re: SSH Client perpetually connecting

Post by ViktorV » Wed 14 Nov 2018 12:01

Unfortunately, we can't reproduce the issue on the latest SecureBridge 8.2.4 version: the code line Log(e.ClassName+ e.Message) was always executed if an error was raised when executing the TSSHClient.Connect method. It is possible that the error will not be raised: when calling the TSSHClient.Connect method, a successful connection to the socket occurs and then the connection is broken - but in this case, after calling TSSHClient.Connect the TSSHClient.Сonnected property will be set to False. To understand the issue cause, we need a test sample where the issue is stably reproduced. As soon as we get such a sample, and if the cause of the issue is in the code of our product, we will try to fix it as soon as possible.
Therefore, please compose a small sample demonstrating the described behavior and send it to us using the contact form http://devart.com/company/contactform.html

ED-Clint
Posts: 36
Joined: Thu 18 May 2017 08:52

Re: SSH Client perpetually connecting

Post by ED-Clint » Mon 19 Nov 2018 09:05

Hi Viktor,

In trying to cut this down to make a simple demo for you I am seeing some odd behaviour, unfortunately not the exact problem raised here. I'll send you the demo project so you can see what I am seeing and if you could give me your thoughts on it I'd appreciate that.

Basically what the demo does is to create an SSH connection and a tunnel, then open and close MySQL connections over the tunnel. There is a meo box that shows the various connect/disconnect events and should simply show a connecting, connected, dosconnecting, disconnected pattern, however I get the odd erroneous disconnected right after a connecting. This may be related to my previous issue in that I am seeing it all hang after the connecting, so maybe the cause of this erroneous disconnected is the issue, whatever it may be.

ViktorV
Devart Team
Posts: 3168
Joined: Wed 30 Jul 2014 07:16

Re: SSH Client perpetually connecting

Post by ViktorV » Thu 22 Nov 2018 13:25

Thank you for the information. We will investigate this behavior of SecureBridge and will inform you about the result.

ED-Clint
Posts: 36
Joined: Thu 18 May 2017 08:52

Re: SSH Client perpetually connecting

Post by ED-Clint » Thu 13 Dec 2018 15:06

Hi Viktor,

Do you have any results as yet?

ViktorV
Devart Team
Posts: 3168
Joined: Wed 30 Jul 2014 07:16

Re: SSH Client perpetually connecting

Post by ViktorV » Fri 14 Dec 2018 10:45

We will continue investigation of the SecureBridge behavior and inform you as soon as we get any results, but we can't provide any time frame at the moment.

ViktorV
Devart Team
Posts: 3168
Joined: Wed 30 Jul 2014 07:16

Re: SSH Client perpetually connecting

Post by ViktorV » Tue 08 Jan 2019 14:29

Please check whether the specified behavior is reproduced when using the latest version of SecureBridge 9.0.1

ED-Clint
Posts: 36
Joined: Thu 18 May 2017 08:52

Re: SSH Client perpetually connecting

Post by ED-Clint » Mon 14 Jan 2019 10:44

Hi Viktor,

I am seeing the same behaviour with SecureBridge 9.0.1. Did you see the behaviour I described and assuming so do you see any difference with 9.0.1 ?

ViktorV
Devart Team
Posts: 3168
Joined: Wed 30 Jul 2014 07:16

Re: SSH Client perpetually connecting

Post by ViktorV » Fri 18 Jan 2019 15:13

We have answered you via e-mail.

Post Reply