psycopg2 copy_from raises asynchronous ProgrammingError with sync connection in celery task

asynchronous celery psycopg2 python sqlbulkcopy

Question

My issue is that I am

ProgrammingError: copy_from cannot be used with an asynchronous callback.

while attemptingcopy_from not using an async connection. It must be made clear that I'm making connections via a celery task. Can someone explain to me how sqlalchemy, celery, or another program makes my psycopg2 connection act async?

conn = psycopg2.connect(con_string)
conn.async
>>0
cur = conn.cursor()    
data = BytesIO()
data.write('\n'.join(['Tom\tJenkins\t37',
                  'Madonna\t\N\t45',
                  'Federico\tDi Gregorio\t\N']))
data.seek(0)
curs.copy_from(data, 'test_copy')
1
2
1/15/2016 2:12:58 PM

Popular Answer

We encountered similar problem in pgcli, where it was discovered that a wait callback was causing the connection to seem to be asynchronous to psycopg2. This aided in

from contextlib import contextmanager

@contextmanager
def _paused_thread():
    try:
        thread = psycopg2.extensions.get_wait_callback()
        psycopg2.extensions.set_wait_callback(None)
        yield
    finally:
        psycopg2.extensions.set_wait_callback(thread)

with _paused_thread():
    cursor.copy_expert('copy mytable to STDOUT', file)
3
6/3/2016 5:25:29 AM


Related Questions





Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow