I applied to get "Developer" privileges to be able to access the Twitter API, after realizing that Twitter does not allow public scraping using simple Python scripts. The API does allow this sort of thing, though it's fairly involved and difficult to do.
The main goal of course is to set up a structure for the students in my Digital Humanities class -- to help them gather data related to hashtags they themselves might want to research. We'll see if that's actually possible -- and assess their interest in stuff like this -- when we start a unit on social media next week.
* * *
First, the process for getting approval was fairly involved -- I had to answer a pretty detailed series of questions about how I'm going to use the access.
Second, once you get approved for access, there's a learning curve with respect to how to use the API. After messing around unsuccessfully with the methods they themselves suggest and their Cloud-based system (using an app called "Postman"), I went to Google and settled on the method outlined here. Essentially, you install Ruby and a Ruby gem called Twurl, which you can run from the Command Line. Then you have to get authorization to scrape using the two API Keys you get when you create a Project in the Developers menu on Twitter API.
After you've gotten authorization by entering the keys, you can run an authorized scrape of Hashtag data using a simple command like this:
twurl "/1.1/search/tweets.json?q=#MyNameIs&result_type=popular" -j > MyNameIs2.json
You can also try a .JSON --> .CSV parser like this one (limit: file has to be less than 1 MB for the free version). Another one I tried is this one; this last one has no limit.
If you convert the .JSON file to .CSV, you should be able to open the file in Microsoft Excel or Google Sheets and just extract the sections of the table that are relevant to you (the ones most relevant to me are the Tweet text and the User name).
More modifiers. You can modify the Twurl command to get more specific about the kind of results you want. For instance, I tried this command to try and get extended tweets and request 100 results at a time:
twurl "/1.1/search/tweets.json?q=#MyNameIs&tweet_mode=extended&count=100" -j > MyNameIs.json
There are more variations here.
* * *
Some of my preliminary results can be viewed at this Google Docs Spreadsheet. I'm also pasting some select results below.
List generated by Twurl:
Asha Rangappa:
"#MyNameIs Asha. It means Hope. But that is my middle name. My first name is Renuka (pronounced RAY-noo-Kah) which h… https://t.co/jyRCjVTx7D"
"#MyNameIs Asha. It means Hope. But that is my middle name. My first name is Renuka (pronounced RAY-noo-Kah) which h… https://t.co/jyRCjVTx7D"
"#MyNameIs Mimi. It’s pronounced MeMe (not MyMy or MiiMii) & it means I’m done worrying about whether older men will… https://t.co/SR8HFVZYAW"
"#MyNameIs Stephanie which most likely means I was born in New Jersey in the 70s to a Mom & generations of women who… https://t.co/sV3nU97YVM"
Meena Harris:
"#MyNameIs Meenakshi. I'm named after the Hindu goddess, as well as my great great grandmother. I come from a long l… https://t.co/2wCQmrboKv"
".#MyNameIs Lea Thompson. It's pronounced LEE Ah. My mom spelled it that way because my dad was stationed in Pearl H… https://t.co/31vEVP09E8"
"#MyNameIs Stephanie which most likely means I was born in New Jersey in the 70s to a Mom & generations of women who… https://t.co/sV3nU97YVM"
Meena Harris:
"#MyNameIs Meenakshi. I'm named after the Hindu goddess, as well as my great great grandmother. I come from a long l… https://t.co/2wCQmrboKv"
".#MyNameIs Lea Thompson. It's pronounced LEE Ah. My mom spelled it that way because my dad was stationed in Pearl H… https://t.co/31vEVP09E8"
#MyNameIs is Krupali which means compassion in Sanskrit. Little white kids used to make fun of my name when I was a kid. Now, a Republican Senator mocks the name of @KamalaHarris. Go to http://iWillVote.com to make a plan to #vote for leaders who respect our values. #iWillVote
Senator Ted Lieu:
"#MyNameIs Ted W. Lieu. The “W” is short for “Win-Ping” which in Mandarin means Cloud of Peace. Also, make sure yo… https://t.co/DknDT2A4BF"
Maya Harris (Kamala's sister)
"Scrolling through #MyNameIs at the end of a long day & it’s giving me life.💜 My name is Maya (magic/illusion) Laks… https://t.co/5v1GtntIAm"
"#mynameis Jahana /juh-ha’-nuh/ (noun): (1) smart, strong, compassionate leader; (2) Congresswoman, teacher, wif… https://t.co/sx1kv7MSW3"
"#MyNameIs Maria de Lourdes Hinojosa Ojeda and it’s all Mexican and catholic and one day I will sign off by saying i… https://t.co/IvVQb8O02X"
Anand Giridaradhas:
"#MyNameIs Anand, which means happiness. We will deal with my last name later. https://t.co/UD6lJXvwiG"
Preet Bharara:
"#MyNameIs Preet, which means love. https://t.co/JSO0zB77aR"
"#MyNameIs Daniel Dae Hyun Kim, or in Korean, 김대현. It means “great and powerful one.” My name is not “Macaca” or “Ku… https://t.co/s8tDiBHMbY"
"#MyNameIs Michelle Wing Kwan & in Chinese pinyin it's pronounced Guan Ying Shan. It means beautiful, strong and sma… https://t.co/VSFo0RzzOj"
"#MyNameIs Eric. My first name in Chinese was “Liang”, which means “bright”. As a child immigrant to the US, I cho… https://t.co/k2fhKpB7ka"
"#MyNameIs Debra. My family comes from Poland and Russia. I stand with #Kamala. #Georgia #DavidPerdue is a DISGRAC… https://t.co/uz4EYPr0gt"
Actor Kal Penn:
"#MyNameIs Kalpen. I started going by Kal Penn to help me get a job & am more than happy to give @SenDavidPerdue som… https://t.co/jaWZIgn7lt"
* * *
List generated by simply Copying and Pasting from Twitter:
Zara Ahmed, DrPH
#MyNameIs Zara. My parents spent a lot of time and energy picking out a name that would be easily pronounceable for non-Indian people. But they shouldn’t have had to do that. It shouldn’t be on POC to shave off pieces of our culture to make life easier for white people.
Ayanna Pressley
US House candidate, MA-7
#MyNameIs Ayanna Soyini Pressley. My mother, may she rest in power, gave me this name which means beautiful flower in Swahili. She told me I’d make history and the world would learn how to say it right.
Nusheen Ameenuddin MD MPH MPA FAAP
#MyNameIs Nusheen Ameenuddin. Nusheen is Persian & it means a sweet dream that comes true. Ameenuddin is Urdu/Arabic & it means trustworthy upholder pf the faith. My names mean something important. I will help you pronounce them. It’s easier than you think.
Ilhan Omar
US House candidate, MN-5
#MyNameIs Ilham, I prefer Ilhan. I never liked the M sound. It means “Inspiration” in Arabic. My father named me Ilham and inspired me to lead a life of service to others. In his honor I am voting for an inspirational ticket over desperate and maddening one.
Ted Lieu
US House candidate, CA-33
#MyNameIs Ted W. Lieu. The “W” is short for “Win-Ping” which in Mandarin means Cloud of Peace. Also, make sure you vote. And if you are voting by mail, don’t forget the stupid secrecy envelope if your state requires it.
Rep. Malcolm Kenyatta
Senator Ted Lieu:
"#MyNameIs Ted W. Lieu. The “W” is short for “Win-Ping” which in Mandarin means Cloud of Peace. Also, make sure yo… https://t.co/DknDT2A4BF"
Maya Harris (Kamala's sister)
"Scrolling through #MyNameIs at the end of a long day & it’s giving me life.💜 My name is Maya (magic/illusion) Laks… https://t.co/5v1GtntIAm"
"#mynameis Jahana /juh-ha’-nuh/ (noun): (1) smart, strong, compassionate leader; (2) Congresswoman, teacher, wif… https://t.co/sx1kv7MSW3"
"#MyNameIs Maria de Lourdes Hinojosa Ojeda and it’s all Mexican and catholic and one day I will sign off by saying i… https://t.co/IvVQb8O02X"
Anand Giridaradhas:
"#MyNameIs Anand, which means happiness. We will deal with my last name later. https://t.co/UD6lJXvwiG"
Preet Bharara:
"#MyNameIs Preet, which means love. https://t.co/JSO0zB77aR"
"#MyNameIs Daniel Dae Hyun Kim, or in Korean, 김대현. It means “great and powerful one.” My name is not “Macaca” or “Ku… https://t.co/s8tDiBHMbY"
"#MyNameIs Michelle Wing Kwan & in Chinese pinyin it's pronounced Guan Ying Shan. It means beautiful, strong and sma… https://t.co/VSFo0RzzOj"
"#MyNameIs Eric. My first name in Chinese was “Liang”, which means “bright”. As a child immigrant to the US, I cho… https://t.co/k2fhKpB7ka"
"#MyNameIs Debra. My family comes from Poland and Russia. I stand with #Kamala. #Georgia #DavidPerdue is a DISGRAC… https://t.co/uz4EYPr0gt"
Actor Kal Penn:
"#MyNameIs Kalpen. I started going by Kal Penn to help me get a job & am more than happy to give @SenDavidPerdue som… https://t.co/jaWZIgn7lt"
* * *
List generated by simply Copying and Pasting from Twitter:
Zara Ahmed, DrPH
#MyNameIs Zara. My parents spent a lot of time and energy picking out a name that would be easily pronounceable for non-Indian people. But they shouldn’t have had to do that. It shouldn’t be on POC to shave off pieces of our culture to make life easier for white people.
Ayanna Pressley
US House candidate, MA-7
#MyNameIs Ayanna Soyini Pressley. My mother, may she rest in power, gave me this name which means beautiful flower in Swahili. She told me I’d make history and the world would learn how to say it right.
Nusheen Ameenuddin MD MPH MPA FAAP
#MyNameIs Nusheen Ameenuddin. Nusheen is Persian & it means a sweet dream that comes true. Ameenuddin is Urdu/Arabic & it means trustworthy upholder pf the faith. My names mean something important. I will help you pronounce them. It’s easier than you think.
Ilhan Omar
US House candidate, MN-5
#MyNameIs Ilham, I prefer Ilhan. I never liked the M sound. It means “Inspiration” in Arabic. My father named me Ilham and inspired me to lead a life of service to others. In his honor I am voting for an inspirational ticket over desperate and maddening one.
Ted Lieu
US House candidate, CA-33
#MyNameIs Ted W. Lieu. The “W” is short for “Win-Ping” which in Mandarin means Cloud of Peace. Also, make sure you vote. And if you are voting by mail, don’t forget the stupid secrecy envelope if your state requires it.
Rep. Malcolm Kenyatta
#MyNameIs Malcolm Kenyatta. My father named me to honor the legacy of two giants in the black diaspora: Malcolm X and Jomo Kenyatta. @sendavidperdue might not care, but my name reminds me everyday the systems of discrimination we must uproot. twitter.com/meenaharris/st…
Meena Harris
@meenaharris
· Oct 17
#MyNameIs Meenakshi. I'm named after the Hindu goddess, as well as my great great grandmother. I come from a long line of strong women who taught me to be proud of my heritage and to demand respect—especially from racist white men like @sendavidperdue who are threatened by us. twitter.com/gauragDC/statu…
Pramila Jayapal
#MyNameIs Pramila. It comes from the Sanskrit word “prem” which means love. The name is constantly mispronounced as is my last name. I only mind that when it is done willfully and continuously. Let’s build an inclusive America. Vote #BidenHarris2020. Our vote, our power.
Bee Nguyen
#MyNameIs Nguyen. I pronounce it “WIN.” Nguyen stems from the Chinese word “ruan” — a string instrument that is plucked. According to the 2010 Census, it’s the 37th most common surname in the U.S. It’s even on a coke bottle! Can @CocaCola put @KamalaHarris on a bottle?
Ro Khanna
US House candidate, CA-17
#MyNameIs Rohit, and my friends call me Ro. It means bright light in Sanskrit. This election, #IWillVote for an inclusive America by voting for @joebiden & @kamalaharris
Gautam Raghavan
My great grandmother’s name was Kamala. Not “Kamala-mala-mala, I don’t know, whatever.” #MyNameIs Gautam. It means bright light. The kind of bright light a Biden-Harris Administration will represent. And that is why #IWillVote. Join me & make a plan at http://IWillVote.com.
Dr. Hiral Tipirneni
US House candidate, AZ-6
#MyNameIs Hiral. It means diamond, bright, full of luster. My mom used to say that they picked it because she saw a bright light reflecting from me & a toughness that made me unbreakable. It’s that strength my mom saw in me that gave me the courage to run for Congress.
Meena Harris
@meenaharris
· Oct 17
#MyNameIs Meenakshi. I'm named after the Hindu goddess, as well as my great great grandmother. I come from a long line of strong women who taught me to be proud of my heritage and to demand respect—especially from racist white men like @sendavidperdue who are threatened by us. twitter.com/gauragDC/statu…
Pramila Jayapal
#MyNameIs Pramila. It comes from the Sanskrit word “prem” which means love. The name is constantly mispronounced as is my last name. I only mind that when it is done willfully and continuously. Let’s build an inclusive America. Vote #BidenHarris2020. Our vote, our power.
Bee Nguyen
#MyNameIs Nguyen. I pronounce it “WIN.” Nguyen stems from the Chinese word “ruan” — a string instrument that is plucked. According to the 2010 Census, it’s the 37th most common surname in the U.S. It’s even on a coke bottle! Can @CocaCola put @KamalaHarris on a bottle?
Ro Khanna
US House candidate, CA-17
#MyNameIs Rohit, and my friends call me Ro. It means bright light in Sanskrit. This election, #IWillVote for an inclusive America by voting for @joebiden & @kamalaharris
Gautam Raghavan
My great grandmother’s name was Kamala. Not “Kamala-mala-mala, I don’t know, whatever.” #MyNameIs Gautam. It means bright light. The kind of bright light a Biden-Harris Administration will represent. And that is why #IWillVote. Join me & make a plan at http://IWillVote.com.
Dr. Hiral Tipirneni
US House candidate, AZ-6
#MyNameIs Hiral. It means diamond, bright, full of luster. My mom used to say that they picked it because she saw a bright light reflecting from me & a toughness that made me unbreakable. It’s that strength my mom saw in me that gave me the courage to run for Congress.
Parag Mehta
#MyNameIs Parag. It means nectar. My mom named me after the hero in a Bengali romance novel she read as a kid. She wanted her son to have a great love story. And I did. Our names matter. They have meaning and beauty.
Dr Shola Mos-Shogbamimu
#MyNameIs AdeÅŸola shortened to Sola which I prefer to spell as Shola because the 'ÅŸ' is the 'sh' sound. I love my name. In Yoruba it means 'crowned with wealth' or 'this crown makes wealth' as my father explained to me Our names matter - Say. My. Name.
#MyNameIs Parag. It means nectar. My mom named me after the hero in a Bengali romance novel she read as a kid. She wanted her son to have a great love story. And I did. Our names matter. They have meaning and beauty.
Dr Shola Mos-Shogbamimu
#MyNameIs AdeÅŸola shortened to Sola which I prefer to spell as Shola because the 'ÅŸ' is the 'sh' sound. I love my name. In Yoruba it means 'crowned with wealth' or 'this crown makes wealth' as my father explained to me Our names matter - Say. My. Name.
Aftab Pureval
#MyNameIs Aftab. It means sunshine. In one generation, my parents have gone from immigrating to this country to watching their son serve as an elected official. I can’t wait to vote for @JoeBiden and @KamalaHarris because they value diversity and believe in our American story.
Linh Nguyen
#MyNameIs Linh Thuy Nguyen. Linh, meaning spirit, soul from Sino-Vietnamese. My dad wanted a traditional name for me to honor our lineage & where we came from. Name pronunciation, and taking the time to do it right, emphasizes safety & belonging, and is a sign of respect.
#MyNameIs Aftab. It means sunshine. In one generation, my parents have gone from immigrating to this country to watching their son serve as an elected official. I can’t wait to vote for @JoeBiden and @KamalaHarris because they value diversity and believe in our American story.
Linh Nguyen
#MyNameIs Linh Thuy Nguyen. Linh, meaning spirit, soul from Sino-Vietnamese. My dad wanted a traditional name for me to honor our lineage & where we came from. Name pronunciation, and taking the time to do it right, emphasizes safety & belonging, and is a sign of respect.