psycopg2 copy_from raises asynchronous ProgrammingError with sync connection in celery task
asynchronous celery psycopg2 python sqlbulkcopy
Question
My problem is I am facing
ProgrammingError: copy_from cannot be used with an asynchronous callback.
while trying to copy_from without async connection. This must be stated, I am creating connection from a celery task. Can someone give me a clue how sqlalchemy or celery or whatever forces my psycopg2 connection behave like async?
conn = psycopg2.connect(con_string)
conn.async
>>0
cur = conn.cursor()
data = BytesIO()
data.write('\n'.join(['Tom\tJenkins\t37',
'Madonna\t\N\t45',
'Federico\tDi Gregorio\t\N']))
data.seek(0)
curs.copy_from(data, 'test_copy')
Popular Answer
We were facing this error in pgcli; in that case it turned out that a wait_callback was making the connection behave as though it were asynchronous from psycopg2's point of view. This helped:
from contextlib import contextmanager
@contextmanager
def _paused_thread():
try:
thread = psycopg2.extensions.get_wait_callback()
psycopg2.extensions.set_wait_callback(None)
yield
finally:
psycopg2.extensions.set_wait_callback(thread)
with _paused_thread():
cursor.copy_expert('copy mytable to STDOUT', file)
Related Questions
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow