streaming_job
=============

.. py:module:: streaming_job


Functions
---------

.. autoapisummary::

   streaming_job.get_config
   streaming_job.write_to_postgres
   streaming_job.main


Module Contents
---------------

.. py:function:: get_config()

.. py:function:: write_to_postgres(batch_df: pyspark.sql.DataFrame, batch_id: int, cfg: dict)

   Utile function to write the batch into the sql table using the connection

   :param batch_df: Spark Dataframe to write into the DB
   :type batch_df: DataFrame
   :param batch_id: a unique identifier for the batch
   :type batch_id: int
   :param cfg: DB connection configuration
   :type cfg: dict

   :returns: execution status
   :rtype: int


.. py:function:: main()