1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Flask - concurrent futures not running as a background process

Discussion in 'Programming/Internet' started by data_garden, Sep 12, 2018.

  1. data_garden

    data_garden Guest

    I am trying to run a Flask app which consists of:

    1. Yielding API requests on the fly
    2. Uploading each request to a SQLalchemy database
    3. Run jobs 1 and 2 as a background process

    For that I have the following code:

    from flask import Flask
    from flask import current_app
    import concurrent.futures
    from concurrent.futures import ThreadPoolExecutor
    import queue

    app = Flask(__name__)
    q = queue.Queue()

    def build_cache():
    # 1. Yielding API requests on the fly
    tracks_and_features = spotify.query_tracks() # <- a generator
    while True:

    def upload_cache(tracks_and_features):
    # 2. Uploading each request to a `SQLalchemy` database
    with app.app_context():


    def cache():
    # 3. Do `1` and `2` as a background process
    with concurrent.futures.ThreadPoolExecutor() as executor:

    future_to_track = {executor.submit(build_cache): 'TRACKER DONE'}

    while future_to_track:
    # check for status of the futures which are currently working
    done, not_done = concurrent.futures.wait(

    # if there is incoming work, start a new future
    while not q.empty():

    # fetch a track from the queue
    track = q.get()

    # Start the load operation and mark the future with its TRACK
    future_to_track[executor.submit(upload_cache, track)] = track
    # process any completed futures
    for future in done:
    track = future_to_track[future]
    data = future.result()
    except Exception as exc:
    print('%r generated an exception: %s' % (track, exc))

    del future_to_track[future]

    return 'Cacheing playlist in the background...'

    All of the above works, BUT NOT AS A BACKGROUND PROCESS. The app halts when cache() is called.

    what am I doing wrong?

    Login To add answer/comment

Share This Page