SqlBulkCopy Tutorial - Knowledge Base (KB)
28 results for: in tag: csv
SqlBulkCopy throws System.FormatException when running WriteToServer(DataTable)
Currently I'm writing a method to read data from a CSV file and import to a SQL table.... DataTable dt = new DataTable(); String line = null; int i = 0; while ((line = reader.ReadLine()) != null) { Strin...
asked by Leo
Uploading a csv file to sql server - Identity problem
Given a column structure in a CSV file of:...First_Name, Last_Name, Date_Of_Birth ...And a SQL Server table with a structure of...ID(PK) | First_Name | Last_Name | Date_Of_Birth ...(Field ID is an Identity with an auto-increment of 1)...How do i arrange i...
asked by MrBliz
How do I read a large file from disk to database without running out of memory
I feel embarrassed to ask this question as I feel like I should already know. However, given I don't....I want to know how to read large files from disk to a database without getting an OutOfMemory exception. Specifically, I need to load CSV (or really ta...
asked by Mr Moose
duplicate products added when importing csv files
I am using the code below to import the csv files onto sqlserver 2008, the issue is , its adding duplicate records onto the table , how can i check to avoid duplicate records import , below is the code which i am using , i have copied from somewhere , and...
asked by Mr A
Handling empty fields in csv during bulk import to database
I'm importing geocode data to our database from a csv file....I've used the following library ...A fast csv reader... to read the csv and then using ...SqlBulkCopy...Here's an example of the data I'm importing..."AB10","1BH","L",0,0,,,20 "AB10","1BR","L...
asked by Neil
how to use ascii character for quote in COPY in cqlsh
I am uploading data from a a big .csv file into Cassandra using copy in cqlsh. I am using cassandra 1.2 and CQL 3.0. However since " is part of my data I have to use some other character for uploading my data, I need to use any extended ASCII characters....
asked by eldho
How to create custom class w/ IDataReader from LumenWorks (Fast CSV Reader) to use SqlBulkCopy while having several different file versions?
I have decided to replace/convert DataTable to IDataReader due to memory issues. ...After quite a while of Google & MSDN searches, I came across this at ...http://www.michaelbowersox.com/2011/12/22/using-a-custom-idatareader-to-stream-data-into-a-databas...
asked by fletchsod
How to add CsvHelper records to DataTable to use for SqlBulkCopy to the database
I am trying to read a CSV file with CsvHelper, load each record into a DataTable, and then use SqlBulkCopy to insert the data into a database table. With the current code, I get an exception when adding a row to the DataTable. The exception is: "Unable to...
asked by Justin Nafe
SqlBulkCopy ColumnMapping Argumentexception
the following code gives me an argumentexception saying, that the sourcecolumn 'e_partnerid', which is clearly in the csv, doesn't exist. I have a feeling that the Delimiter isn't set right or something like that and I tried changing the connectionstring ...
asked by OhSnap
SqlBulkCopy - empty datarows in db2table
now I finally got to write the rows of my csv (somewhat) into the table. It does add the 7 rows of the csv, but without data. The only thing it sets is the id, but that's autoincremented anyways. The data out of the csv doesn't arrive in the table.... sta...
asked by OhSnap
SqlBulkCopy ColumnMapping multiple columns
I'm using ...SqlBulkCopy... to import CSV data to a database, but I'd like to be able to combine columns. For instance, lets say I have a firstname and lastname column in my CSV. Through my UI, I'd like the user to be able to choose firstname + lastname t...
asked by roryok
value getting changed during SqlBulkCopy
I am trying to upload some data from csv file to sql server using SqlBulkCopy The problem here is that the values get changed automatically eg 0.9824 becomes 0.982400000095367...Here is the code that I am using...using (SqlBulkCopy bulkCopy = new SqlBulkC...
asked by xoanon
sqlBulk insert C#
I have a page where I want to upload a CSV file from my computer to database on the server and I have my opentext that looks like the following...using (StreamReader sr = File.OpenText(@"c:\users\workstationUsername\FileName.csv"))...This works fine on my...
asked by Master Page
Select specific Fields to output to CSV - Postgresql
I have a database from which i have to export to a csv/text file the ...longtitude... and ...lantitude... in this form:...25.767368,-80.18930 34.088808,-118.40612 40.727093,-73.97864 # Lat,Long ...The attributes in my database are in a correct format and...
asked by The Mini John
sqlbulkcopy - When converting string to DateTime
I am using this code to insert a csv file to my database:... private void InsertDataIntoSQLServerUsingSQLBulkCopy(DataTable csvFileData) { using (SqlConnection dbConnection = new SqlConnection(ConnectionString)) { dbConn...
asked by TeaAnyOne
CSV to SQL - Add 1 day to a Date in a datacolumn
I am trying to add 1 day to all dates that are in a certain datacolumn ['RecordAddedDate']...csvData.Columns.AddRange(new DataColumn[3] { new DataColumn("Manufacturer", typeof(string)), new DataColumn("SupplierCode", typeof(string)), ...
asked by TeaAnyOne
Set constant value to some columns in SqlBulkCopy
I'm trying transfer data from ..."csv"... file to SQL Database. Is it possible to map some fields by default vale which not exsits in ..."csv"... file? Something like shown below: ...bulkCopy.ColumnMappings.Add("Destenition_column_name", "constant_value")...
asked by user3818229
C# - Importing a CSV where empty strings do not get sent through bulk import to database
So I have a database of site location information which contains 27 columns. I have coded a bulk import feature that takes a CSV file and lets you modify information. Everything works well when the CSV file contains information in every column, but when t...
asked by ouro
SQL Server - can't import value 0 or 1 to bit column
This seems like a duplicate question. But as I searched & viewed the related questions & answers, i still can't find the answer....I am writing a loader program which imports data records from CSV file to a database table. Here is the demo....The database...
asked by yyou
What is the proper way to insert millions of records from a csv file into a SQL Server database?
I have to insert 40 million records from a ....csv... file into a database and below is the process I followed....Windows Service 1:...Reading the CSV...validating the records...inserting valid records into success table (intermediate table) using ...SqlB...
asked by sainath
Page 1 of 2