Best practice for transferring a large amount of data. Using Stored Procedure or SQL Bulk Copy

How to use SQL bulk copy to retrieve the IDs of added data records

I'm currently doing something like this:...DataTable objects = new DataTable(); DataColumn keyColumn = new DataColumn("name", typeof(string)); DataColumn versionColumn = new DataColumn("version", typeof(int)); versionColumn.DefaultValue = iVersionID; objects.Columns.Add(keyColumn); objects.Columns.Add(versionColumn); foreach (KeyValuePair<string,...

Is it possible to transfer data across servers using sql bulk copy?

Yes. In fact, in the demo code shown ...here..., they copy to a table within the same database.

SQL Bulk Copy to SQL Server with Data Modifications in Excel

I am reading up on ...SQLBulkCopy... and would like to use it to import thousands of rows from an Excel document to SQL Server. I keep reading articles that do it directly without modifying the data. I need to do some modifications and validations on the data in the Excel document before doing the ...SQLBulkCopy.... Is it possible to do this? I ...

Using file stream, sql bulk copy will not upload data to the database.

Simplify your code; there are plenty of ways to get the csv into a datatable - see ...How to read a CSV file into a .NET Datatable... for example. In your example above, you do not appear to be closing the stream after you have finished reading it... add sr.Close() before the return, or better still wrap the declaration in a using() statement:... ...

Splitting a huge table into two and bulk copying using SQL Server and C#.net

I would create it in one table and create well defined indexes. ...Consider a filter index for the flag column. This is done with a where clause in t-sql and the filter page in ssms. ...http://msdn.microsoft.com/en-us/library/cc280372.aspx

Access double to SQL decimal using SQL Bulk Copy

Your problem is that floating-point types like ...Double... often cannot store numbers ...exactly..., even values which seem to be quite "simple" numbers to us humans. This can be demonstrated by typing the following into the Immediate Window of the VBA editor in Access:...?(CDbl(81.60) * CDbl(100)) - CDbl(8160) ...The result returned is...-5.6843...

Missing Data after insert into my end sql table using Sql Bulk Copy

you need to look at the data in Access, and find/fix those bad column values, or handle them in your TSQL after the export. One option to debug - replace temp table with real table, replace datetime datatype with string, do your export, then query the table to find the bad values so you know what kind of data to look for. After that, you can swit...

How to send large amount of Data To SqlServer Stored Procedure

I think you are seeking for ...Table-Valued Parameters.... There is enough documentation how to create TVP type in the database and use it inside a stored procedure. When setting up a ...SqlParameter..., you can supply ...IEnumerable<SqlDataRecord>... which can easily be provided by a C# iterator function directly reading sequentially CSV file data...

Why do I get high fragmentation when using SqlBulkCopy to move large amounts data between databases?

Its hard to tell without more data but I bet that your timestamp being a clustered index is causing this. Try sorting the data by this field before sending to the output table.

Using SQL Bulk Copy to insert ~3.7 million rows throws OutOfMemory exception

Thank you all for your response....I figured out what was the problem. Firstly, it didn't occur when calling the SQL Bulk Copy, which I assumed it could have been because, in previous run, I was able to run few 1000 less records than the current one, so I thought the SQL Bulk Copy had limitations or need to settings in order to handle large number ...