What is the most efficient approach to load a large quantity of data without using a database engine.
I use SqlBulkCopy when utilizing SQL Server, but I wish to disconnect from it.
There is no method to achieve this that is independent of database engines. Every DB server operates differently (e.g. bcp for Sybase).
If that's what you're looking for, you may want to clarify your query. There could be some third-party product that can perform the job for you, but it will just be a wrapper over server-specific techniques below.
NOTICE: Since INSERTs are MUCH slower than native bulk inserts owing to being logged, performing a lot of INSERTs like Mark advised is DEFINITELY NOT the same (unlike bcp).
bulk insert that is database independent? Not conceivable.
My best guess is to write an insert script with the following lines:
INSERT INTO TableName (...) VALUES (...);
It will efficiently compress, therefore gzip would be a good option.
However, it's usually best to stay with the instructions designed specifically for databases. "Database independent" is often simply another way to say "slow."