I am trying to insert a fairly large dataset into typedb. I have inserted the entities first and now inserting their relationships. I am using Python API for this but the insertion is taking more than a whole day to insert.
I have looked into parallelization and batch insertion concepts but still not sure how to incorporate that.
I am attaching the code snippet for insertion. Can someone review it and tell me how to incorporate batch and parallelization into that?
with TypeDB.core_client(“localhost:1729”,10) as client:
with client.session(“KG”, SessionType.DATA) as session:
for x in Relationlines:
if len(x.strip()) != 0 :
with session.transaction(TransactionType.WRITE) as write_transaction:
insert_iterator = write_transaction.query().insert(x)
print(‘Inserted’ + x )