Add key to each of the asynced API data-CodePudding

symbols = ['A','B','C']
datalist1a = []
datalist1b = []

def get_tasks1(session1):
    tasks1 = []
    for symbol in symbols:
        tasks1.append(session1.get(url1.format(symbol)))
    return tasks1

async def run_tasks1():
    session1 = aiohttp.ClientSession()
    tasks1 = get_tasks1(session1)
    responses1 = await asyncio.gather(*tasks1)
    for response1 in responses1:
        datalist1a.append(await response1.json())
        datalist1b = sum(datalist1a, [])
    await session1.close()

datalist1a gives the following output with 2 data (can be changed by the API parameters) per symbol

[
    [
        [1642086000000, '0.031', '0.032', 1642086299999, '0'],
        [1642086300000, '0.033', '0.036', 1642086599999, '0']
    ],
    [   [1642086000000, '0.101', '0.111', 1642086299999, '0'],
        [1642086300000, '0.091', '0.099', 1642086899999, '0']
    ],
    [   [1642086000000, '0.431', '0.461', 1642086299999, '0'],
        [1642086300000, '0.461', '0.481', 1642086899999, '0']
]

Then I apply sum(datalist1a, []), and the new output is as follows:

[
    [1642086000000, '0.031', '0.032', 1642086299999, '0'],
    [1642086300000, '0.033', '0.036', 1642086599999, '0'],
    [1642086000000, '0.101', '0.111', 1642086299999, '0'],
    [1642086300000, '0.091', '0.099', 1642086899999, '0'],
    [1642086000000, '0.431', '0.461', 1642086299999, '0'],
    [1642086300000, '0.461', '0.481', 1642086899999, '0']
]

The final format is good to be inserted/updated to the database.

However the API does not include the symbols in the data.

I need to insert each symbol to the corresponding list. As some data might be missing, comes null or gives errors, I think it would not be a good idea to append the symbols list to the final data.

What I need:

[
    ['A', 1642086000000, '0.031', '0.032', 1642086299999, '0'],
    ['A', 1642086300000, '0.033', '0.036', 1642086599999, '0'],
    ['B', 1642086000000, '0.101', '0.111', 1642086299999, '0'],
    ['B' 1642086300000, '0.091', '0.099', 1642086899999, '0'],
    ['C', 1642086000000, '0.431', '0.461', 1642086299999, '0'],
    ['C' 1642086300000, '0.461', '0.481', 1642086899999, '0']
]

CodePudding user response：

You could just add another coroutine that does the HTTP call and return both the response and the symbol. So in code, I would think of something like

def get_tasks1(session1):
    async def get_response_and_symbol(symbol:str) -> tuple[Response, str]:
        response = await session1.get(url1.format(symbol))
        return response, symbol
    return [get_response_and_symbol(symbol) for symbol in symbols]
...

this should give you a symbol and the response. You just have to make sure further down how to handle that correctly for your needs.

CodePudding user response：

Thank you, it worked like a charm. The only thing is

for i in responses1:
datalist1a.extend(sum(([i[1]], await i[0].json()), []))

gives the output:

[
'A',
[1642086000000, '0.031', '0.032', 1642086299999, '0'],
[1642086300000, '0.033', '0.036', 1642086599999, '0'],
'B',
[1642086000000, '0.101', '0.111', 1642086299999, '0'],
[1642086300000, '0.091', '0.099', 1642086899999, '0'],
'C',
[1642086000000, '0.431', '0.461', 1642086299999, '0'],
[1642086300000, '0.461', '0.481', 1642086899999, '0']
]

I'll try to figure it someway