This README_SongOfOurselves.txt file was generated on 2023-10-30 by CAITLIN S. MATHEIS
Edited by Data Curator Sarah Reiff Conell 



GENERAL INFORMATION

1. Title of Dataset: Songs of Ourselves: The Circulations and Citations of Nineteenth-Century American Poetry on Twitter

2. Author Information
	A. Principal Investigator Contact Information
		Name: Caitlin Matheis, M.A., M.L.I.S.
		Institution: University of Iowa
		Email: caitlin-matheis@uiowa.edu
		ORCID: https://orcid.org/0009-0004-6480-1218

	B. Associate or Co-investigator Contact Information
		Name: Dr. Micah Bateman
		Institution: University of Iowa
		Email: micah-bateman@uiowa.edu
		ORCID: https://orcid.org/0000-0003-3575-2432

3. Data collection period: 2022-11-18 through 2023-01-18

4. Funded by: School of Library and Information Science, University of Iowa https://ror.org/036jqmy94


SHARING/ACCESS INFORMATION

The data are licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) and may be cited according to standard citation practices from the Nineteenth-Century Data Collective with Matheis cited as the first author. 


DATA & FILE OVERVIEW

1. File List: 
|-- README_SongOfOurselves.txt
|-- DataEssay_SongOfOurselves.pdf
|-- CompletedTwitterSearchList.pdf
|-- CompletedTwitterSearchList.csv
|-- DataCSVs
|	|-- alcottlm-count.csv
|	|-- allenea-count.csv
|	|-- baldricht-count.csv
|	|-- barlowj-count.csv
|	|-- belljm-count.csv
|	|-- bibbe-count.csv
|	|-- bryansm-count.csv
|	|-- bryantwc-count.csv
|	|-- bushbanksow-count.csv
|	|-- busyheadj-count.csv
|	|-- campbellag-count.csv
|	|-- campbellch-count.csv
|	|-- campbellje-count.csv
|	|-- caryp-count.csv
|	|-- chandlerem-count.csv
|	|-- channingwe-count.csv
|	|-- childlm-count.csv
|	|-- chiversth-count.csv
|	|-- cookert-count.csv
|	|-- cranchcr-count.csv
|	|-- cranes-count.csv
|	|-- crayh-count.csv
|	|-- crosbyf-count.csv
|	|-- davidsonlm-count.csv
|	|-- davisdv-count.csv
|	|-- dickinsone-count.csv
|	|-- douglassf-count.csv
|	|-- dunbarpl-count.csv
|	|-- duncandc-count.csv
|	|-- duncanjc-count.csv
|	|-- eastmandr-count.csv
|	|-- eastmaneg-count.csv
|	|-- eellsjm-count.csv
|	|-- emersone-count.csv
|	|-- emersonrw-count.csv
|	|-- fawcette-count.csv
|	|-- fieldsa-count.csv
|	|-- fieldsjt-count.csv
|	|-- folsomi-count.csv
|	|-- fordhammw-count.csv
|	|-- fortunett-count.csv
|	|-- freemanmw-count.csv
|	|-- fullerm-count.csv
|	|-- garlandhh-count.csv
|	|-- gilmancp-count.csv
|	|-- goodichsg-count.csv
|	|-- grimkec-count.csv
|	|-- griswoldrc-count.csv
|	|-- gummerefb-count.csv
|	|-- guyjh-count.csv
|	|-- harperfew-count.csv
|	|-- hawthornen-count.csv
|	|-- hayneph-count.csv
|	|-- heardjd-count.csv
|	|-- hedgefh-count.csv
|	|-- higginsontw-count.csv
|	|-- holmesow-count.csv
|	|-- hooperes-count.csv
|	|-- hortongm-count.csv
|	|-- howejw-count.csv
|	|-- howellswd-count.csv
|	|-- jacksonhh-count.csv
|	|-- jewettso-count.csv
|	|-- keyfs-count.csv
|	|-- lazaruse-count.csv
|	|-- longfellowhw-count.csv
|	|-- lowelljr-count.csv
|	|-- lowellm-count.csv
|	|-- mayc-count.csv
|	|-- melvilleh-count.csv
|	|-- menardjw-count.csv
|	|-- menkenai-count.csv
|	|-- metoxenm-count.csv
|	|-- moorecc-count.csv
|	|-- nelsonad-count.csv
|	|-- parleyp-count.csv
|	|-- percyf-count.csv
|	|-- piattd-count.csv
|	|-- piattjj-count.csv
|	|-- piatts-count.csv
|	|-- platoa-count.csv
|	|-- poeea-count.csv
|	|-- poseya-count.csv
|	|-- readtb-count.csv
|	|-- reasond-count.csv
|	|-- ridgejr-count.csv
|	|-- robinsonea-count.csv
|	|-- rogersep-count.csv
|	|-- sandburgc-count.csv
|	|-- schoolcraftjj-count.csv
|	|-- schoolcrafth-count.csv
|	|-- sigourneyl-count.csv
|	|-- simpsonjm-count.csv
|	|-- sixkillers-count.csv
|	|-- smitheo-count.csv
|	|-- spoffordhp-count.csv
|	|-- stedmaned-count.csv
|	|-- stoddardrh-count.csv
|	|-- stowehb-count.csv
|	|-- tappancs-count.csv
|	|-- taylorb-count.csv
|	|-- teconeeskee-count.csv
|	|-- terryl-count.csv
|	|-- thoreauhd-count.csv
|	|-- tillmankd-count.csv
|	|-- tooquastee-count.csv
|	|-- truths-count.csv
|	|-- tsoleohwho-count.csv
|	|-- tuckerme-count.csv
|	|-- veryj-count.csv
|	|-- walkerw-count.csv
|	|-- wardsg-count.csv
|	|-- whartone-count.csv
|	|-- whitfieldjm-count.csv
|	|-- whitmanaa-count.csv
|	|-- whitmanw-count.csv
|	|-- whittierjg-count.csv
|	|-- willarde-count.csv
|	|-- willisnp-count.csv



2. Relationship between files, if important:
 
Elizabeth Akers Allen published many of her early poems under the pseudonym "Forence Percey. Searches for tweet counts were completed for both names and are listed and contained in the dataset as files <eaallen-count.csv> and <fpercy-count.csv>.

Sarah Piatt is known both by her married name and maiden name, Sarah Morgan Bryan. Searches for tweet counts were completed for both names and are listed and contained in the dataset as files <spiatt-count.csv> and <smbryan-count.csv>.

Too-Qua-Stee is also known as Dewitt Clinton Duncan. Searches for tweet counts were completed for both names and are contained in the dataset as files <tooquastee-count.csv> and <dcduncan-count.csv>.

Samuel Griswold Goodrich was better known by his pseudonym Peter Parley. Searches were completed for both names and are contained in the dataset as files <sggoodrich-count.csv> and <pparley-count.csv>.


3. Are there multiple versions of the dataset?

Our datasets reflect the number of extant tweets from 2022-2023 fully citing a given author’s name as of the date the data were pulled. This means that historical data could change on a daily basis depending on attrition—the deletion of tweets (now posts) or the suspension of user accounts—and our counts would not reflect that attrition without updates, which are impossible in the Musk era without researcher access to full-archive data. This is currently the only version of the dataset.


METHODOLOGICAL INFORMATION

The initial data were pulled throughout the fall of 2022 and  in January 2023, using a Python notebook that Walsh adapted from Ed Summers, which utilizes a Twitter-scraping tool, twarc2, from the Documenting the Now collective. We used Bateman’s full-archive researcher access to authenticate the data pulls, an affordance of Twitter’s Academic Research product track that the platform launched in January 2021 to allow academic researchers easier access to all of the Twitter API’s endpoints before the access was terminated by new leadership in 2023. The results of a search in the Python notebook show the number of tweets plus retweets per day that match the exact string of text searched by pulling from a “tweet counts” API endpoint. Our notebook strictly retrieves circulation counts (“get count”), so our data do not contain any specific tweets, any personal information about Twitter users, or references to their account information and thus aligns with the terms of condition for researcher access to Twitter’s full archive as well as preserves users’ right to be forgotten. Each author csv was downloaded directly from the notebook; no changes or alterations were made to the data after it was scraped. 

Our searches currently include names of poets as they are popularly known or referred to in the anthologies cited. However, in instances where a writer is primarily known by or published under multiple names, including pseudonyms, multiple searches were conducted. Names were taken from indexes and entered using all lowercase letters, included dashes and periods where necessary, and excluded hashtags. This means that our data are limited to the full citation of poets, which includes misattributions and homonymous citations and excludes variable or pseudonymous attribution (to, for instance, “W. Whitman”). Counts of “Walt Whitman” and “Emily Dickinson” likely refer to the nineteenth-century poets, or else to places and organizations named after them, such as Walt Whitman High School. But string searches for “William Walker” likely include many more references to persons who are not the nineteenth-century poet intended.

Link to "tweet count" API endpoints: <https://developer.twitter.com/en/docs/twitter-api/tweets/counts/introduction>
Link to twarc on GitHub: <https://github.com/ericscartier/twarc>
Link to Melanie Walsh's GitHub repository for collecting Twitter data with twar<https://github.com/melaniewalsh/Collecting-Twitter-Data>



DATA-SPECIFIC INFORMATION FOR: Author CSVs

1. Number of variables: 3

2. Number of cases/rows: One column for each variable; Each file begins on 2006-03-21, the day that Twitter was officially launched and the first tweet was sent, and includes data for each day and hour up until the data was pulled. For example, if data was pulled on 2022-11-15 at 11 AM CST, the data included would be from midnight on 2006-03-21 and would include counts until 2022-11-15 at 11 AM CST.

3. Variable List: 
<start> - includes the starting date and time frame included in that row's day count
<end> - includes the ending date and time frame included in that row's day count
<day_count> - the number of tweets that include string of text searched in the time frame specified in <start> and <end>



DATA-SPECIFIC INFORMATION FOR: Complete Twitter Search CSV

1. Number of variables: 5

2. Number of cases/rows: 119

3. Variable List: 
<author name> - the name of the author the search is being completed for
<author search string> - the string of text searched to find counts
<date of search> - the date the search was completed
<time search was completed> - the time that the notebook finished retrieving the counts
<other names> - included only if applicable, this field indicates names from the same author that are also in the dataset, for example, if an author was also commonly known under a pen name, they appear in the dataset twice under both names