sign in | join about | help | sitemap
Connecting Tech Pros Worldwide
=?Utf-8?B?SmFzb24=?='s Avatar

How can I avoid ftp timout issues?


Question posted by: =?Utf-8?B?SmFzb24=?= (Guest) on August 27th, 2008 02:25 PM
Hello, can someone please suggest to me how I can keep my ftp connection from
timing out after a large file download?

I am using SSIS and I have tried multiple ways of ftp'ing the files in my
directory to my local hard drive, but the tasks keep failing after a large
file is downloaded. The error I'm getting is:

An error occurred in the requested FTP operation. Detailed error
description: Receiving file "usfile.zip".
The operation timed out

The file is about 120MB.

I tried using the ftp task, as well as writing (reverse engineering) an ftp
script task using FtpWebRequest.Create and both methods work as long as the
files are small.

It looks like the ftp timeout property has to be set between 1 and 300, so
even though I am explicitly assigning a timeout property of 0 in my
connection via script, it is still timing out.

Any suggestions are much appreciated. I've been struggling with this issue
for about a week now.

Thanks!

Jason
http://www.lonestarfinancing.com

7 Answers Posted
Steve Gerrard's Avatar
Guest - n/a Posts
#2: Re: How can I avoid ftp timout issues?

Jason wrote:
Quote:
Originally Posted by
Hello, can someone please suggest to me how I can keep my ftp
connection from timing out after a large file download?
>
I am using SSIS and I have tried multiple ways of ftp'ing the files
in my directory to my local hard drive, but the tasks keep failing
after a large file is downloaded. The error I'm getting is:
>
An error occurred in the requested FTP operation. Detailed error
description: Receiving file "usfile.zip".
The operation timed out
>
The file is about 120MB.
>
I tried using the ftp task, as well as writing (reverse engineering)
an ftp script task using FtpWebRequest.Create and both methods work
as long as the files are small.
>
It looks like the ftp timeout property has to be set between 1 and
300, so even though I am explicitly assigning a timeout property of 0
in my connection via script, it is still timing out.
>
Any suggestions are much appreciated. I've been struggling with this
issue for about a week now.
>


In addition to a timeout on the client, there is also a timeout on the server.
You may be hitting an FTP connection timeout on the server, which can be
configured on the server end of things. I have done that for HTTP, not FTP, but
Googling it should help. It may just be a property on the FTP site in IIS.

There are some places where a timeout of 0 (no limit) is not accepted, and it
reverts to the default setting. You might also try a large value instead, like
3600 seconds.


=?Utf-8?B?SmFzb24=?='s Avatar
=?Utf-8?B?SmFzb24=?= August 27th, 2008 04:45 PM
Guest - n/a Posts
#3: Re: How can I avoid ftp timout issues?

Hello Steve,

I checked microsoft's site, and you can see here that the timeout setting
must be between 1 and 300:
http://technet.microsoft.com/en-us/...ut(SQL.90).aspx.

I tried changing it to 0 or a higher value, but it doesn't look it actually
accepts any value other than between 1 and 300.


I've been googling this issue for a week now, and I'm not sure what to do.
Do you know how I can tell if the problem is on my side or on the server?

Thanks,

Jason
http://www.lonestarfinancing.com



"Steve Gerrard" wrote:
Quote:
Originally Posted by
Jason wrote:
Quote:
Originally Posted by
Hello, can someone please suggest to me how I can keep my ftp
connection from timing out after a large file download?

I am using SSIS and I have tried multiple ways of ftp'ing the files
in my directory to my local hard drive, but the tasks keep failing
after a large file is downloaded. The error I'm getting is:

An error occurred in the requested FTP operation. Detailed error
description: Receiving file "usfile.zip".
The operation timed out

The file is about 120MB.

I tried using the ftp task, as well as writing (reverse engineering)
an ftp script task using FtpWebRequest.Create and both methods work
as long as the files are small.

It looks like the ftp timeout property has to be set between 1 and
300, so even though I am explicitly assigning a timeout property of 0
in my connection via script, it is still timing out.

Any suggestions are much appreciated. I've been struggling with this
issue for about a week now.

>
In addition to a timeout on the client, there is also a timeout on the server.
You may be hitting an FTP connection timeout on the server, which can be
configured on the server end of things. I have done that for HTTP, not FTP, but
Googling it should help. It may just be a property on the FTP site in IIS.
>
There are some places where a timeout of 0 (no limit) is not accepted, and it
reverts to the default setting. You might also try a large value instead, like
3600 seconds.
>
>
>

J.B. Moreno's Avatar
Guest - n/a Posts
#4: Re: How can I avoid ftp timout issues?

In article <F1A78297-A372-4C6E-8D8A-8619F4446878@microsoft.com>, Jason
<jason2004@online.nospamwrote:
Quote:
Originally Posted by
Hello, can someone please suggest to me how I can keep my ftp connection from
timing out after a large file download?


FTP downloads take place on two connections, most likely your command
connection is timing out for lack of use...you might give it a kick and
send it a command every once in a while.

--
J.B. Moreno
=?Utf-8?B?SmFzb24=?='s Avatar
=?Utf-8?B?SmFzb24=?= August 27th, 2008 05:15 PM
Guest - n/a Posts
#5: Re: How can I avoid ftp timout issues?

Hello JB,

I am new to programming, but below is the code I'm using. It works until a
large file (120MB) is downloaded. Once it downloads, no other files in the
directory are downloaded, and I get an error msg saying my connection timed
out.


Can you send me an example of what I should do based on my code below?

------------
'And we obtain our file(s):

ftp.ReceiveFiles(sbDownloadFiles.ToString().Substr ing(0, _

sbDownloadFiles.ToString.Length - 1).Split("|".ToCharArray()), _

Dts.Variables("FtpDestination").Value.ToString, True, False)
-------------

Also, do I need to set KeepAlive to True? I thought is defaulted to True.

Here is my code:

------------------------------------


Public Sub Main()

Dim ftp As FtpClientConnection

Dim retrieveFiles As Boolean = False

Dim ftpWeb As FtpWebRequest

Try

ftpWeb = CType(FtpWebRequest.Create("ftp://" +
Dts.Variables("FTPURL").Value.ToString + _

"/" + Dts.Variables("FTPDirectory").Value.ToString), FtpWebRequest)

ftpWeb.Credentials = New
NetworkCredential(Dts.Variables("FTPUser").Value.ToString,
Dts.Variables("FTPPassword").Value.ToString())

ftpWeb.Method = WebRequestMethods.Ftp.ListDirectoryDetails

'Using fileReader As StreamReader = File.OpenText(baseFileName)

Dim srResponse As New StreamReader(ftpWeb.GetResponse().GetResponseStrea m())

Dim readLine As String

Dim stringComponents() As String

Dim sbDownloadFiles As New StringBuilder()

Dim fileEnding As String = Dts.Variables("FtpFileEnding").Value.ToString

Dim ignoreDate As Boolean =
Convert.ToBoolean(Dts.Variables("IgnoreDate").Value.ToString)

While Not srResponse.EndOfStream()

readLine = srResponse.ReadLine()

If readLine.Trim().EndsWith(fileEnding) Then

stringComponents = readLine.Split(" ".ToCharArray())

'Since the file is updated at least once a month, we just check the day:

If ignoreDate Or Math.Abs(DateTime.Now.Day -
Int32.Parse(stringComponents(17))) < 2 Then

'This file has been modified within the last day or so...

retrieveFiles = True

sbDownloadFiles.Append(stringComponents(19) + "|")

End If

End If

End While

srResponse.Close()

srResponse = Nothing

ftpWeb = Nothing

If retrieveFiles Then

'Now we need to build our FTP connection:

Dim fiRemoteFile As FileInfo

Dim ftpConnectionManager As ConnectionManager

ftpConnectionManager = Dts.Connections.Add("FTP")

ftpConnectionManager.Properties("ServerName").SetValue(ftpConnectionManager,
Dts.Variables("FTPURL").Value)

ftpConnectionManager.Properties("ServerPort").SetValue(ftpConnectionManager,
Dts.Variables("FTPPort").Value)

ftpConnectionManager.Properties("ServerUserName").SetValue(ftpConnectionManager, Dts.Variables("FTPUser").Value)

ftpConnectionManager.Properties("ServerPassword").SetValue(ftpConnectionManager, Dts.Variables("FTPPassword").Value)

ftpConnectionManager.Properties("Timeout").SetValue(ftpConnectionManager,
Dts.Variables("FTPTimeout").Value)

ftpConnectionManager.Properties("ChunkSize").SetValue(ftpConnectionManager,
Dts.Variables("FtpChunkSize").Value)

ftpConnectionManager.Properties("Retries").SetValue(ftpConnectionManager,
Dts.Variables("FTPRetries").Value)

ftp = New FtpClientConnection(ftpConnectionManager.AcquireCo nnection(Nothing))

'Next we connect to the ftp server

ftp.Connect()

ftp.SetWorkingDirectory(Dts.Variables("FTPDirectory").Value.ToString)

'And we obtain our file(s):

ftp.ReceiveFiles(sbDownloadFiles.ToString().Substr ing(0, _

sbDownloadFiles.ToString.Length - 1).Split("|".ToCharArray()), _

Dts.Variables("FtpDestination").Value.ToString, True, False)

End If

'If we did not download anything new there is nothing to process...

Dts.Variables("ContinueProcessing").Value = retrieveFiles

Dts.TaskResult = Dts.Results.Success

Catch exError As Exception

Throw exError

Finally

If Not ftpWeb Is Nothing Then

ftpWeb = Nothing

End If

If Not ftp Is Nothing Then

ftp.Close()

End If

End Try

End Sub

-------------------------------------


Thanks,
Jason



"J.B. Moreno" wrote:
Quote:
Originally Posted by
In article <F1A78297-A372-4C6E-8D8A-8619F4446878@microsoft.com>, Jason
<jason2004@online.nospamwrote:
>
Quote:
Originally Posted by
Hello, can someone please suggest to me how I can keep my ftp connection from
timing out after a large file download?

>
FTP downloads take place on two connections, most likely your command
connection is timing out for lack of use...you might give it a kick and
send it a command every once in a while.
>
--
J.B. Moreno
>

J. Moreno's Avatar
Guest - n/a Posts
#6: Re: How can I avoid ftp timout issues?

=?Utf-8?B?SmFzb24=?= <jason2004@online.nospamwrote:
Quote:
Originally Posted by
Hello JB,
>
I am new to programming, but below is the code I'm using. It works until
a large file (120MB) is downloaded. Once it downloads, no other files in
the directory are downloaded, and I get an error msg saying my connection
timed out.
>
Can you send me an example of what I should do based on my code below?


You probably can't do it with FtpWebRequest, it's not meant to be a full
blown FTP client. If it doesn't work under your circumstances (say your
connecting to an IBM Mainframe, or apparently downloading more than xMB)
then it doesn't work and you'll have to write your own FTP code, or get a
library that does it (not hard to do), and use that instead.

The program that I maintain that does FTP uses a clsFTP converted to VB in
2002 by Vick S.:
<http://www.tek-tips.com/viewthread.cfm?qid=1074156&page=8>.

Changing it back to C# isn't too hard (I did it while trying to add SSL
support).

--
J. Moreno
Jialiang Ge [MSFT]'s Avatar
Jialiang Ge [MSFT] August 28th, 2008 08:25 AM
Guest - n/a Posts
#7: Re: How can I avoid ftp timout issues?

Hello Jason,

Apart from J. Moreno and Steve's solutions, I'd suggest your trying posting
this SQL Server DTS question to the microsoft.public.sqlserver.dts
newsgroup.

Regards,
Jialiang Ge (jialge@online.microsoft.com, remove 'online.')
Microsoft Online Community Support

=================================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
Join Bytes!.

This posting is provided "AS IS" with no warranties, and confers no rights.
=================================================

kimiraikkonen's Avatar
Guest - n/a Posts
#8: Re: How can I avoid ftp timout issues?

On Aug 27, 4:14 pm, Jason <jason2...@online.nospamwrote:
Quote:
Originally Posted by
Hello, can someone please suggest to me how I can keep my ftp connection from
timing out after a large file download?
>
I am using SSIS and I have tried multiple ways of ftp'ing the files in my
directory to my local hard drive, but the tasks keep failing after a large
file is downloaded. The error I'm getting is:
>
An error occurred in the requested FTP operation. Detailed error
description: Receiving file "usfile.zip".
The operation timed out
>
The file is about 120MB.
>
I tried using the ftp task, as well as writing (reverse engineering) an ftp
script task using FtpWebRequest.Create and both methods work as long as the
files are small.
>
It looks like the ftp timeout property has to be set between 1 and 300, so
even though I am explicitly assigning a timeout property of 0 in my
connection via script, it is still timing out.
>
Any suggestions are much appreciated. I've been struggling with this issue
for about a week now.
>
Thanks!
>
Jasonhttp://www.lonestarfinancing.com


That may not do the trick but you may try, have you tried:

My.Computer.Network.DownloadFile(....) method and increase
"connectionTimeout" parameter to what you want.(It represents 32-bit
integer to specify).

See this:
http://msdn.microsoft.com/en-us/library/ack30t8y.aspx

Onur Güzel
 
Not the answer you were looking for? Post your question . . .
197,001 members ready to help you find a solution.
Join Bytes.com

What is Bytes?

We are a network of experts and professionals in IT and software development that help one another with answers to tough questions and share insights. Get the best answers to your questions from over 197,001 network members.
Post your question now . . .
It's fast and it's free

Popular Articles

Top Community Contributors