2026-06-24 15:55:46 | INFO     | Pipeline Start Date: 2026-06-23
2026-06-24 15:55:46 | INFO     | ================================================================================
2026-06-24 15:55:46 | INFO     | Hello from data-move Python data pipeline!
2026-06-24 15:55:46 | INFO     | Pipeline Run Date: 2026-06-23
2026-06-24 15:55:46 | INFO     | Connecting to databases...
2026-06-24 15:55:48 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x000002C8151D4730>
2026-06-24 15:55:52 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x000002C8162F0B90>
2026-06-24 15:55:52 | INFO     | Database connections established
2026-06-24 15:55:52 | INFO     | Collecting MIDs for: 2026-06-23
2026-06-24 15:55:52 | INFO     | Found 863 MIDs
2026-06-24 15:55:52 | WARNING  | Pipeline failed. Retry 1/3. Error: 'tables'
2026-06-24 15:55:57 | INFO     | ================================================================================
2026-06-24 15:55:57 | INFO     | Hello from data-move Python data pipeline!
2026-06-24 15:55:57 | INFO     | Pipeline Run Date: 2026-06-23
2026-06-24 15:55:57 | INFO     | Connecting to databases...
2026-06-24 15:55:58 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x000002C81600B550>
2026-06-24 15:55:58 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x000002C8162F5A90>
2026-06-24 15:55:59 | INFO     | Database connections established
2026-06-24 15:55:59 | INFO     | Collecting MIDs for: 2026-06-23
2026-06-24 15:55:59 | INFO     | Found 863 MIDs
2026-06-24 15:55:59 | WARNING  | Pipeline failed. Retry 2/3. Error: 'tables'
2026-06-24 15:56:04 | INFO     | ================================================================================
2026-06-24 15:56:04 | INFO     | Hello from data-move Python data pipeline!
2026-06-24 15:56:04 | INFO     | Pipeline Run Date: 2026-06-23
2026-06-24 15:56:04 | INFO     | Connecting to databases...
2026-06-24 15:56:04 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x000002C8162F59A0>
2026-06-24 15:56:05 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x000002C8162F68A0>
2026-06-24 15:56:05 | INFO     | Database connections established
2026-06-24 15:56:05 | INFO     | Collecting MIDs for: 2026-06-23
2026-06-24 15:56:05 | INFO     | Found 863 MIDs
2026-06-24 16:10:30 | INFO     | Pipeline Start Date: 2026-06-23
2026-06-24 16:10:30 | INFO     | ================================================================================
2026-06-24 16:10:30 | INFO     | Hello from data-move Python data pipeline!
2026-06-24 16:10:30 | INFO     | Pipeline Run Date: 2026-06-23
2026-06-24 16:10:30 | INFO     | Connecting to databases...
2026-06-24 16:10:31 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x000001B29EE8C470>
2026-06-24 16:10:32 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x000001B29FEBCA50>
2026-06-24 16:10:32 | INFO     | Database connections established
2026-06-24 16:10:32 | INFO     | Collecting MIDs for: 2026-06-23
2026-06-24 16:10:33 | INFO     | Found 863 MIDs
2026-06-24 16:10:33 | WARNING  | Pipeline failed. Retry 1/3. Error: 'tables'
2026-06-24 16:10:38 | INFO     | ================================================================================
2026-06-24 16:10:38 | INFO     | Hello from data-move Python data pipeline!
2026-06-24 16:10:38 | INFO     | Pipeline Run Date: 2026-06-23
2026-06-24 16:10:38 | INFO     | Connecting to databases...
2026-06-24 16:10:38 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x000001B29FBB7550>
2026-06-24 16:10:39 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x000001B29FECD9A0>
2026-06-24 16:10:40 | INFO     | Database connections established
2026-06-24 16:10:40 | INFO     | Collecting MIDs for: 2026-06-23
2026-06-24 16:10:40 | INFO     | Found 863 MIDs
2026-06-24 16:10:40 | WARNING  | Pipeline failed. Retry 2/3. Error: 'tables'
2026-06-24 16:10:45 | INFO     | ================================================================================
2026-06-24 16:10:45 | INFO     | Hello from data-move Python data pipeline!
2026-06-24 16:10:45 | INFO     | Pipeline Run Date: 2026-06-23
2026-06-24 16:10:45 | INFO     | Connecting to databases...
2026-06-24 16:10:45 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x000001B29FECD8B0>
2026-06-24 16:10:46 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x000001B29FECE7B0>
2026-06-24 16:10:46 | INFO     | Database connections established
2026-06-24 16:10:46 | INFO     | Collecting MIDs for: 2026-06-23
2026-06-24 16:10:47 | INFO     | Found 863 MIDs
2026-06-24 16:12:02 | INFO     | Pipeline Start Date: 2026-06-23
2026-06-24 16:12:02 | INFO     | ================================================================================
2026-06-24 16:12:02 | INFO     | Hello from data-move Python data pipeline!
2026-06-24 16:12:02 | INFO     | Pipeline Run Date: 2026-06-23
2026-06-24 16:12:02 | INFO     | Connecting to databases...
2026-06-24 16:12:02 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x00000258DBAE05D0>
2026-06-24 16:12:03 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x00000258DCB8CA50>
2026-06-24 16:12:04 | INFO     | Database connections established
2026-06-24 16:12:04 | INFO     | Collecting MIDs for: 2026-06-23
2026-06-24 16:12:04 | INFO     | Found 863 MIDs
2026-06-24 16:12:05 | INFO     | Pipeline Run Date: 2026-06-23
2026-06-24 16:12:05 | INFO     | ================================================================================
2026-06-24 16:12:05 | INFO     | Processing Table-:OQaD | Table type -:FACT | fetcht by-:run_date | operation-:INSERT
2026-06-24 16:12:05 | INFO     | Fetching  Data  from sql server  for table-: OQaD  ..............
2026-06-24 16:12:05 | INFO     | Exists: True
2026-06-24 16:12:05 | INFO     | Path: D:\z\airflow-project\src\sql\fact\OQaD.sql
2026-06-24 16:12:05 | INFO     | Fetching data for 863 EMPIDs
2026-06-24 16:12:05 | INFO     | Fetching OQaD data for run_date=2026-06-23
2026-06-24 16:12:11 | INFO     | fn name is fetch_OQad ------Fetched 458 rows
2026-06-24 16:12:11 | INFO     | Fetched total row  -: 458 from sql server  for table-:OQaD ...........!!!
2026-06-24 16:12:11 | INFO     | Fetched 458 rows
2026-06-24 16:12:11 | INFO     | _ _ _ _ Deleting Data  from ClickHouse for OQaD _ _ _ _
2026-06-24 16:12:11 | INFO     | _ _ _  _Inserting data into clickhouse db from sql server_ _ _ _
2026-06-24 16:12:11 | ERROR    | Failed processing table OQaD
Traceback (most recent call last):
  File "D:\z\airflow-project\elt.py", line 243, in elt
    load_to_clickhouse(
    ~~~~~~~~~~~~~~~~~~^
        client=client,
        ^^^^^^^^^^^^^^
        table_name=table_name,
        ^^^^^^^^^^^^^^^^^^^^^^
        df=df,
        ^^^^^^
    )
    ^
  File "D:\z\airflow-project\clickhouse_task\load_table.py", line 28, in load_to_clickhouse
    chunk_df = df.iloc[start:end]
               ^^^^^^^
AttributeError: 'DataFrame' object has no attribute 'iloc'
2026-06-24 16:12:11 | WARNING  | Pipeline failed. Retry 1/3. Error: 'DataFrame' object has no attribute 'iloc'
2026-06-24 16:12:16 | INFO     | ================================================================================
2026-06-24 16:12:16 | INFO     | Hello from data-move Python data pipeline!
2026-06-24 16:12:16 | INFO     | Pipeline Run Date: 2026-06-23
2026-06-24 16:12:16 | INFO     | Connecting to databases...
2026-06-24 16:12:17 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x00000258DCB6C650>
2026-06-24 16:12:17 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x00000258DCB99A90>
2026-06-24 16:12:18 | INFO     | Database connections established
2026-06-24 16:12:18 | INFO     | Collecting MIDs for: 2026-06-23
2026-06-24 16:12:18 | INFO     | Found 863 MIDs
2026-06-24 16:12:18 | INFO     | Pipeline Run Date: 2026-06-23
2026-06-24 16:12:18 | INFO     | ================================================================================
2026-06-24 16:12:18 | INFO     | Processing Table-:OQaD | Table type -:FACT | fetcht by-:run_date | operation-:INSERT
2026-06-24 16:12:18 | INFO     | Fetching  Data  from sql server  for table-: OQaD  ..............
2026-06-24 16:12:18 | INFO     | Exists: True
2026-06-24 16:12:18 | INFO     | Path: D:\z\airflow-project\src\sql\fact\OQaD.sql
2026-06-24 16:12:18 | INFO     | Fetching data for 863 EMPIDs
2026-06-24 16:12:18 | INFO     | Fetching OQaD data for run_date=2026-06-23
2026-06-24 16:12:24 | INFO     | fn name is fetch_OQad ------Fetched 458 rows
2026-06-24 16:12:24 | INFO     | Fetched total row  -: 458 from sql server  for table-:OQaD ...........!!!
2026-06-24 16:12:24 | INFO     | Fetched 458 rows
2026-06-24 16:12:24 | INFO     | _ _ _ _ Deleting Data  from ClickHouse for OQaD _ _ _ _
2026-06-24 16:12:24 | INFO     | _ _ _  _Inserting data into clickhouse db from sql server_ _ _ _
2026-06-24 16:12:24 | ERROR    | Failed processing table OQaD
Traceback (most recent call last):
  File "D:\z\airflow-project\elt.py", line 243, in elt
    load_to_clickhouse(
    ~~~~~~~~~~~~~~~~~~^
        client=client,
        ^^^^^^^^^^^^^^
        table_name=table_name,
        ^^^^^^^^^^^^^^^^^^^^^^
        df=df,
        ^^^^^^
    )
    ^
  File "D:\z\airflow-project\clickhouse_task\load_table.py", line 28, in load_to_clickhouse
    chunk_df = df.iloc[start:end]
               ^^^^^^^
AttributeError: 'DataFrame' object has no attribute 'iloc'
2026-06-24 16:12:24 | WARNING  | Pipeline failed. Retry 2/3. Error: 'DataFrame' object has no attribute 'iloc'
2026-06-24 16:12:29 | INFO     | ================================================================================
2026-06-24 16:12:29 | INFO     | Hello from data-move Python data pipeline!
2026-06-24 16:12:29 | INFO     | Pipeline Run Date: 2026-06-23
2026-06-24 16:12:29 | INFO     | Connecting to databases...
2026-06-24 16:12:30 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x00000258DCB9A120>
2026-06-24 16:12:30 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x00000258DCB9A8A0>
2026-06-24 16:12:31 | INFO     | Database connections established
2026-06-24 16:12:31 | INFO     | Collecting MIDs for: 2026-06-23
2026-06-24 16:12:31 | INFO     | Found 863 MIDs
2026-06-24 16:12:31 | INFO     | Pipeline Run Date: 2026-06-23
2026-06-24 16:12:31 | INFO     | ================================================================================
2026-06-24 16:12:31 | INFO     | Processing Table-:OQaD | Table type -:FACT | fetcht by-:run_date | operation-:INSERT
2026-06-24 16:12:31 | INFO     | Fetching  Data  from sql server  for table-: OQaD  ..............
2026-06-24 16:12:31 | INFO     | Exists: True
2026-06-24 16:12:31 | INFO     | Path: D:\z\airflow-project\src\sql\fact\OQaD.sql
2026-06-24 16:12:31 | INFO     | Fetching data for 863 EMPIDs
2026-06-24 16:12:31 | INFO     | Fetching OQaD data for run_date=2026-06-23
2026-06-24 16:12:43 | INFO     | fn name is fetch_OQad ------Fetched 458 rows
2026-06-24 16:12:43 | INFO     | Fetched total row  -: 458 from sql server  for table-:OQaD ...........!!!
2026-06-24 16:12:43 | INFO     | Fetched 458 rows
2026-06-24 16:12:43 | INFO     | _ _ _ _ Deleting Data  from ClickHouse for OQaD _ _ _ _
2026-06-24 16:12:43 | INFO     | _ _ _  _Inserting data into clickhouse db from sql server_ _ _ _
2026-06-24 16:12:43 | ERROR    | Failed processing table OQaD
Traceback (most recent call last):
  File "D:\z\airflow-project\elt.py", line 243, in elt
    load_to_clickhouse(
    ~~~~~~~~~~~~~~~~~~^
        client=client,
        ^^^^^^^^^^^^^^
        table_name=table_name,
        ^^^^^^^^^^^^^^^^^^^^^^
        df=df,
        ^^^^^^
    )
    ^
  File "D:\z\airflow-project\clickhouse_task\load_table.py", line 28, in load_to_clickhouse
    chunk_df = df.iloc[start:end]
               ^^^^^^^
AttributeError: 'DataFrame' object has no attribute 'iloc'
2026-06-24 16:17:01 | INFO     | Pipeline Start Date: 2026-06-23
2026-06-24 16:17:01 | INFO     | ================================================================================
2026-06-24 16:17:01 | INFO     | Hello from data-move Python data pipeline!
2026-06-24 16:17:01 | INFO     | Pipeline Run Date: 2026-06-23
2026-06-24 16:17:01 | INFO     | Connecting to databases...
2026-06-24 16:17:02 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x0000018D3B1F05D0>
2026-06-24 16:17:03 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x0000018D3C29CA50>
2026-06-24 16:17:04 | INFO     | Database connections established
2026-06-24 16:17:04 | INFO     | Collecting MIDs for: 2026-06-23
2026-06-24 16:17:04 | INFO     | Found 863 MIDs
2026-06-24 16:17:04 | INFO     | Pipeline Run Date: 2026-06-23
2026-06-24 16:17:04 | INFO     | ================================================================================
2026-06-24 16:17:04 | INFO     | Processing Table-:OQaD | Table type -:FACT | fetcht by-:run_date | operation-:INSERT
2026-06-24 16:17:04 | INFO     | Fetching  Data  from sql server  for table-: OQaD  ..............
2026-06-24 16:17:04 | INFO     | Exists: True
2026-06-24 16:17:04 | INFO     | Path: D:\z\airflow-project\src\sql\fact\OQaD.sql
2026-06-24 16:17:04 | INFO     | Fetching data for 863 EMPIDs
2026-06-24 16:17:04 | INFO     | Fetching OQaD data for run_date=2026-06-23
2026-06-24 16:17:10 | INFO     | fn name is fetch_OQad ------Fetched 458 rows
2026-06-24 16:17:10 | INFO     | Fetched total row  -: 458 from sql server  for table-:OQaD ...........!!!
2026-06-24 16:17:10 | INFO     | Fetched 458 rows
2026-06-24 16:17:10 | INFO     | _ _ _ _ Deleting Data  from ClickHouse for OQaD _ _ _ _
2026-06-24 16:17:11 | INFO     | _ _ _  _Inserting data into clickhouse db from sql server_ _ _ _
2026-06-24 16:17:11 | INFO     | Inserted rows 0 to 458
2026-06-24 16:17:11 | INFO     | OQaD: inserted 458 rows into ClickHouse
2026-06-24 16:17:11 | INFO     | OQaD loaded successfully (458 rows)
2026-06-24 16:17:11 | INFO     | ================================================================================
2026-06-24 16:17:11 | INFO     | Pipeline Completed Successfully
2026-06-24 16:17:11 | INFO     | ================================================================================
2026-06-24 16:17:11 | INFO     | Pipeline completed successfully. pipeline_trigeered_on_date=2026-06-24last_successful_run_date=2026-06-23
2026-06-24 16:17:31 | INFO     | Pipeline Start Date: 2026-06-23
2026-06-24 16:17:31 | INFO     | ================================================================================
2026-06-24 16:17:31 | INFO     | Hello from data-move Python data pipeline!
2026-06-24 16:17:31 | INFO     | Pipeline Run Date: 2026-06-23
2026-06-24 16:17:31 | INFO     | Connecting to databases...
2026-06-24 16:17:31 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x0000022F667F05D0>
2026-06-24 16:17:33 | INFO     | <sqlalchemy.engine.cursor.CursorResult object at 0x0000022F678CCA50>
2026-06-24 16:17:33 | INFO     | Database connections established
2026-06-24 16:17:33 | INFO     | Collecting MIDs for: 2026-06-23
2026-06-24 16:17:34 | INFO     | Found 863 MIDs
2026-06-24 16:17:34 | INFO     | Pipeline Run Date: 2026-06-23
2026-06-24 16:17:34 | INFO     | ================================================================================
2026-06-24 16:17:34 | INFO     | Processing Table-:OQaD | Table type -:FACT | fetcht by-:run_date | operation-:INSERT
2026-06-24 16:17:34 | INFO     | Fetching  Data  from sql server  for table-: OQaD  ..............
2026-06-24 16:17:34 | INFO     | Exists: True
2026-06-24 16:17:34 | INFO     | Path: D:\z\airflow-project\src\sql\fact\OQaD.sql
2026-06-24 16:17:34 | INFO     | Fetching data for 863 EMPIDs
2026-06-24 16:17:34 | INFO     | Fetching OQaD data for run_date=2026-06-23
2026-06-24 16:17:40 | INFO     | fn name is fetch_OQad ------Fetched 458 rows
2026-06-24 16:17:40 | INFO     | Fetched total row  -: 458 from sql server  for table-:OQaD ...........!!!
2026-06-24 16:17:40 | INFO     | Fetched 458 rows
2026-06-24 16:17:40 | INFO     | _ _ _ _ Deleting Data  from ClickHouse for OQaD _ _ _ _
2026-06-24 16:17:41 | INFO     | _ _ _  _Inserting data into clickhouse db from sql server_ _ _ _
2026-06-24 16:17:41 | INFO     | Inserted rows 0 to 458
2026-06-24 16:17:41 | INFO     | OQaD: inserted 458 rows into ClickHouse
2026-06-24 16:17:41 | INFO     | OQaD loaded successfully (458 rows)
2026-06-24 16:17:41 | INFO     | ================================================================================
2026-06-24 16:17:41 | INFO     | Pipeline Completed Successfully
2026-06-24 16:17:41 | INFO     | ================================================================================
2026-06-24 16:17:41 | INFO     | Pipeline completed successfully. pipeline_trigeered_on_date=2026-06-24last_successful_run_date=2026-06-23