Home > Enterprise >  How can I extract bytes type JSON data from an API connection, and convert selected columns to a pyt
How can I extract bytes type JSON data from an API connection, and convert selected columns to a pyt

Time:01-19

The following is my code with an API connection:

import http.client

conn = http.client.HTTPSConnection("instagram47.p.rapidapi.com")

headers = {
    'x-rapidapi-host': "instagram47.p.rapidapi.com",
    'x-rapidapi-key': "fbac45a2a1mshd1552a6a4b5c40dp11f0fejsn87c5e6ebf342"
    }


conn.request("GET", "/user_following?userid=38500199922", headers=headers)

res = conn.getresponse()
data = res.read()

print(data.decode("utf-8")) ##This is byte data. 
d_string=json.dumps(d) #converting to strings using json.dumps
print(d_string)

Result:

{"body": {"page_size": 200, "next_max_id": "50", "status": "ok", "big_list": true, "users": [{"is_private": true, "pk": 10119571306, "full_name": "geraline lol \ud83e\uddd1\ud83c\udffd\u200d\ud83c\udfa4", "profile_pic_id": "2714545912561066205_10119571306", "username": "g3raline__", "has_anonymous_profile_picture": false, "is_favorite": false, "latest_reel_media": 0, "follow_friction_type": 0, "has_highlight_reels": false, "profile_pic_url": "https://scontent-mxp2-1.cdninstagram.com/v/t51.2885-19/s150x150/260402731_244731964418125_2728204070967694144_n.jpg?_nc_ht=scontent-mxp2-1.cdninstagram.com&_nc_cat=101&_nc_ohc=yFi0ME-BwLoAX9VB2ws&edm=ALB854YBAAAA&ccb=7-4&oh=00_AT_hG7G8rRZ8kXgBjvTUXDmsKY0MdTkf-XxJXMZsmbLRWA&oe=61EE3BA7&_nc_sid=04cb80", "is_verified": false}, {"is_private": true, "pk": 29469548019, "full_name": "", "profile_pic_id": "2742223021579729541_29469548019", "username": "melannyski", "has_anonymous_profile_picture": false, "is_favorite": false, "latest_reel_media": 0, "follow_friction_type": 0, "has_highlight_reels": false, "profile_pic_url": "https://scontent-mxp2-1.cdninstagram.com/v/t51.2885-19/s150x150/271273719_272733284842985_2676050159817911915_n.jpg?_nc_ht=scontent-mxp2-1.cdninstagram.com&_nc_cat=105&_nc_ohc=YYVyklHjfsgAX_eQwZc&edm=ALB854YBAAAA&ccb=7-4&oh=00_AT_TMiNQTrq3xuFWg4WIdxyv_MRfdxSwjsnKUplA1FWu5w&oe=61ED7336&_nc_sid=04cb80", "is_verified": false}, {"is_private": false, "pk": 53892051, "full_name": "Courtney Dickson", "profile_pic_id": "2733057419432607865_53892051", "username": "dirty_dickson1", "has_anonymous_profile_picture": false, "is_favorite": false, "latest_reel_media": 0, "follow_friction_type": 0, "has_highlight_reels": false, "profile_pic_url": "https://scontent-mxp2-1.cdninstagram.com/v/t51.2885-19/s150x150/269483477_210224671299081_9007717381559573256_n.jpg?_nc_ht=scontent-mxp2-1.cdninstagram.com&_nc_cat=109&_nc_ohc=j-SUMBR2fWYAX-TR5Bz&edm=ALB854YBAAAA&ccb=7-4&oh=00_AT8gtIqlMNabJb0c-vQkDcSVyyl9o0nErEdhaRzOp5jmig&oe=61EDB143&_nc_sid=04cb80", "is_verified": false}]}, "status": "Success", "statusCode": 404}

Desired Result:
I want a list of all the username and full_name from users like this list in a python dataframe:

username full_name
dirty_dickson1 Courtney Dickson
Some Username Some Full Name

CodePudding user response:

data = json.loads(data.decode("utf-8"))
df = pd.DataFrame([{"username": u["username"], "full_name": u["full_name"]} for u in data["body"]["users"]])
  •  Tags:  
  • Related