I wish to transfer a lot of data to the SQL Server from a.Net-based application. This data records are arriving from a CSV file, hence a stored procedure on the SQL Server side should be used to process them. SqlBulkCopy, which I am aware of, writes straight to tables. this indicates that I cannot use it. I'm speaking from records that total over 100,000,000. This much information will overwhelm my memory. Therefore, my ideal question is: Is there a way to stream data from.Net to a stored procedure? Any assistance would be appreciated.
You seem to be looking for Parameters with a Table Value. The steps for creating a TVP type in a database and using it in a stored procedure are sufficiently documented. When establishing a
, you may provide
This is simply given by a C# iterator function that systematically reads CSV file data records one by one, converts each record's data, and produces a corresponding result for each record.
. The data will still be bulk-inserted into a tempdb on the SQL Server side before being sent to the stored procedure, but at least on the client side, no memory is required to retain all of that data.