Topic Modeling

Final Reflections

~ Annie Swafford ~ 1 Comment

It’s difficult for me to narrow down how much I’ve learned this semester about Digital Humanities. I guess I’ll choose my good examples from my favorite topics: archives, wordclouds, and topic modeling:

With archives like the Old Bailey, data can be easily accessible online. So so so much data, including books, photos of paintings, sound bits, video, and et cetera. It’s not just digitally scanned documents anymore. You no longer have to travel to some dark basement or well established college in England to see the original paper documents– they are scanned for you, ready to be read. Although physicality is still important, digital archives offer ways to the general public to access once hidden and/or difficult to study materials. Problems concerning this access and what gets put up online are certainly an issue, but digital archives allow scholars and non scholars alike to access things… Which is pretty neat.

Old Bailey Archive Home Screen
Old Bailey Archive Home Screen

The Old Bailey Proceeding site is easy to navigate, offering plenty of instructive videos on the search functions. Huge plus, especially given how much info there is. It also provides illustrations pertaining to the Old Bailey’s history, in paintings or photographic form. The graphic design choices remain uniform and pertinent to the topics at hand. If you are interested, the site allows you to look at original copies of some proceedings. Also, all the data is cited, a number one rule. Continue reading →

Topic Modeling Analysis

~ Annie Swafford

From these topic modeling graphs, trends in the Sherlock Holmes stories as well as the real world can be seen.  It is safe to say that this was the popular culture back in the late 1800s/early 1900s just from seeing the themes within the story.  I thought it was interesting to find the relationships with real world events.

Screen Shot 2015-04-03 at 2.08.40 PM

The first chart shows “crime”, “crime scene”, “murder”, “family and relationships” and “investigation”.  There are a couple of large spikes for family and relationships, especially in the 1920s, although a quick google search leaves me empty handed.  Crime also shows a spike in the early 1920s as well, and this could be because of The Red Scare, which was not exclusive to the United States.  During this several high profile cases in the United States such as, Sacco and Vanzetti as well as the Scopes Monkey Trial have occurred. By this time, news sources in Great Britain would have got word of these cases. The other three topics are very related to crime in itself.

Screen Shot 2015-04-03 at 2.02.08 PM

With the next chart, which shows “finance” and “foreign affairs”, there is one large spike for foreign affairs on September 1st, 1917.  The Great War was still going on, and this was the year the United States entered the war.  Also, Germany has declared unrestrictive submarine warfare several months earlier.  Russia’s position in the war was being questioned as Bolsheviks started to gain more control in Russia, starting with the abdication of Tsar Nicholas II in March of 1917 as well as continuing riots in the country.  Finance unfortunately does not receive the same attention that foreign affairs has been receiving.

Screen Shot 2015-04-03 at 2.57.56 PM

With the last chart, dealing with “smoking”, “residential streets” and “transportation”.  A large spike in residential streets is seen on January 1st, 1904.  In this year, road infrastructure is still in its infancy, roads were still poorly made, cars were not as widespread and modern traffic laws have not been drafted yet.  What is quite strange is that transportation does not see as large of a spike even in 1908 when the Ford Motor Company introduced the Model T, which has quickly become the most popular car around the world, beating British brands such as Austin, Rolls-Royce and Bentley.

Historical Interrelation: Words and War

~ Annie Swafford

 

Sir Arthur Conan Doyle penned all the Sherlock Holmes stories between 1891 and 1927, creating a literary legend that would not soon be forgotten. By using topic modeling techniques and some fancy algorithms, we can investigate the potential relevance of word usage in his stories.

The Great War was a momentous event that almost exactly bisected Doyles’ creations, so I will view my ten topics through this lens. I found an interesting website for historical background on London during this time period, which helped me to identify significant events.

Estate vs Business

Screen Shot 2015-04-01 at 2.39.52 PM

In contrasting estate and business, I noticed a spike in the former before 1905 and again before 1910. This could have been because British colonies had large tea estates in India. However, estate crashed back down after 1910 and business led throughout the rest of the time period. It is possible that the industrial growth of London led to this change, and both words are lower during the war and flu pandemic of 1918.

Search vs Case

Screen Shot 2015-04-01 at 2.40.18 PM

Regarding search and case, there is a slight rise in each during the war years. Once again, the flu of 1918 and the peak of both words during the 1910-1920 decade may involve correlation rather than causation. The term search may also have increased during the war because soldiers could be missing in action.

Crime vs Reasoning 

Screen Shot 2015-04-01 at 2.41.01 PM

Reasoning was mentioned more than crime prior to 1915, but the use of crime skyrocketed after this, calming back down in the 1920’s. This fits well with the suffrage movement and trade unions growing, as this disrupted established society. From the 20th century London website:

The suffragettes, the Irish ‘Home Rule’ movement and trade unions all agitated for change, sometimes with violence. In 1918 some political demands were met through the Representation of the People Act, which gave the vote to working men and women over 30.

Appearance vs Expression

Screen Shot 2015-04-01 at 2.41.19 PM

Appearance quite possibly became less important than expression after the war, due to the realities it forced upon the people of London. This trend continued through the 20’s, as the growth of jazz may have led to expression becoming more common.

Silent Reflection vs Public

Screen Shot 2015-04-01 at 2.41.36 PM

Silent reflection had an interesting spike in 1908, and it dropped precipitously, becoming equal to public by 1914. It is possible that the Alien Act impacted this word usage, as many immigrants tried to come to London during this period. Perhaps many Londoners had thoughts about the impact on their society, but the war decreased their time for such thoughts.

While I am not certain about these linguistic developments, I feel topic modeling could be an important tool to help scholars revisit the past, specifically helpful in distinguishing how history affects word usage.

Lauren Gao’s: Topic Modeling II

~ Annie Swafford

After performing last week’s topic modeling on all 56 Sherlock Holmes short stories, 10 out of the 100 topics generated from last week were put into Google’s Fusion Tables to check for trends in the 10 particular topics of our choice. I chose to mainly look at the time period from January 1892 to July 1893 being that it contained a high concentration of published Sherlock Holmes stories.

The first two topics I looked at and compared were,

Murder and Villains

Screenshot (56)

Continue reading →

Topic Modeling Graph Results

~ Annie Swafford

I wasn’t sure how to label in Google Fusion tables(oops), but in my graphs the X axis represents the publication year and the Y axis represents theme frequency. Overall, I liked thinking about the graph results and musing over what the data might represent.

Gun: There was a large increase in this topic from December 1st 1893 to October 12th 1893. In 1893, The Final Problem was published. Although Holmes dies (insert massive question mark here) in the story, it isn’t gun related. He plummets to his death (insert another massive question mark here) at Reichenbach Falls with Moriarty. However, he is beaten with a police baton, so maybe my topic is faulty. The topic drops the next year, rises again in 1904, and then falls until 1911. After this, the graph experiences spikes in 1917, 1922, and 1925. I looked up guns in Victorian London using victorianlondon.org, and found an entry detailing a gun involved murder from 1876. Given the later dates, and presuming that I didn’t mess up to topic, maybe it’s that guns became more available, and recognized in crime stories.

Gun topic
Gun topic

Continue reading →

Topic Modeling

~ Annie Swafford

Iterations: 1500

Topics: 20

Topics Printed: 20

  1. ESTATE: house road passed round side place walked carriage garden left master dog horse hall drive path led ground walk standing
  2. CRIME: man young found inspector house father colonel dead police heard death body attention son crime evidence murder returned dangerous hopkins
  3. PUBLIC: street found home back station train lord james baker minutes st occurred waiting reached cab hours police late order town
  4. SEARCH: thought time make give made leave knew great find back hear place things doubt bring chance lost position fellow danger
  5. EXPRESSION: cried face turned back hands instant hand suddenly head moment voice words sprang forward eyes fell appeared feet lips threw
  6. APPEARANCE: man eyes face black looked red dark white figure deep hair thin features hat heavy drawn tall appearance blue sharp
  7. REASONING: clear mind point reason question person find matter idea make means absolutely secret presence impossible save excellent aware explanation sign
  8. CASE: interest sherlock case facts strange remarkable friend singular account london cases arthur nature problem extraordinary details public effect give find
  9. SILENT REFLECTION: holmes chair sat gave mrs companion fire fresh visitor rose pipe easy start table glanced silence cold silent horror change
  10. BUSINESS: business london money men papers set office answered letters brother made hundred work address man considerable great company west mycroft

DeFranco_Topic Modeling Assignment

~ Annie Swafford

Iterations: 2000
Number of Topic Words: 15
Number of Topics:150

Body/Posture: face eyes turned caught stood looked sunk glimpse mouth staring breast sat chin sight covered

Attack: man found dead body blood knife struck blow weapon fell wound stick head picked wounded

Crime: crime murder night appeared committed scene charge criminal motive violence arrest discovered lucas tragedy police

Smoking: pipe sat fire lit silence tobacco smoke cigar opposite smoking cigarette smoked peculiar thoughts handed

Family/Household: family england people real year high children live friends folk history shows household governess gather

Writing a Letter: paper read note written wrote handed sheet letter writing post write slip pen tossed printed

Neighborhood: house passed garden door walk gate lane cottage walking park windows dark grounds road knocked

Clothing/Wardrobe: black coat dressed hat st clair broad cap wore dress collar den eye trousers coloured

Thought: mind clear idea make remember observed vague effort easily forced absurd draw suspected memory false

Marriage: woman wife husband love life loved knew married girl women nature marriage marry power lover

Topic Modeling

~ Annie Swafford

The settings I have used were 100 topics, 5,000 iterations and 30 topic words.

Smoking- chair sat fire pipe rose glanced smoke arm tobacco visitor cigar lit asked laughed silence smoking opposite laid back smoked walk armchair leaning dropped seat comfortable waved cigarette details heartily

Residential Streets- house road side place walked drive miles front led hall high stopped standing building gate windows grounds park trap houses mile yards trees low reached cottage narrow row upper lane

Family/Relationships- woman lady husband wife love young loved beautiful girl marriage daughter married nature women maid fiend de seldom marry blame lovely lover hated mistress loves physical hatred listen merville hilda

Crime (general)- crime murder death night reason police account charge evidence appeared committed proved violence arrest remained motive attempt criminal murdered terrible conduct lucas inquest attention constable probable attempted hideous strong event

Finance- money hundred business pounds thirty thousand large pay milverton worth price fifty sum letters check ten paid firm offer terms named ruin bank work city client advance bag cost market

Transportation- back carriage past drove hurried companion journey round pulled save started reach town reached follow cab drive imagine class early wheels immediately curve van stepped holmes rattled swiftly luggage coachman.

Crime(murder)-back left met father knew death poor sister returned died mother fate witness alive called make swear struck terrible unhappy enemies offered jump faithful reply cry success interview impress absent

Crime Scene- body lay found shot head revolver dead blood weapon wound floor bullet pistol lying handle finally hurt close clothes cried fatal die fired powder wounded brains sight inflicted surgeon horribly

Investigation- case watson effect remarked complete follow conclusion knowledge simple methods form result greatest reasoning work art produced credit observation formed deduction occasionally occur cases working touch step poison drug telegraph

Foreign affairs- english american time von gentleman german bork fancy brother british agents greek london added prisoner occasion england country melas secretary book exclamation living people interpreter car admiration mister worst main

Lauren Gao’s Topic Modeling

~ Annie Swafford

Using Mallet’s Topic Modeling program, DHM293 ran all 56 Sherlock Holmes short stories with various settings. Changing the number of iterations and topics, I finally settled on settings of 100 topics and 2000 iterations, with 20 words in each topic.

1) Murder in Sherlock Holmes

crime death police murder reason charge scene tragedy night committed arrest violence evidence murdered motive constable caused suspicion escape attempt

2) Watson

watson dr doctor friend means surprised matter natural blessington amberley patient disease days medical continued knowing reasons armstrong trevelyan brougham

3) Men in Sherlock Holmes Stories

face man eyes dark thin tall expression figure features looked beard voice middle manner handsome gray clean age huge fierce

4) Women in Sherlock Holmes

woman wife husband love life knew girl loved married lady women rich daughter soul beautiful power nature beauty marriage young

5) Transportation in Sherlock Holmes

home minutes cab waiting heard wait glad ten ha walking twenty church quiet send reach talking feel driven long drove

6) Deducing in Sherlock Holmes

case facts points explanation fact simple theory admit investigation give solution problem confess correct present obvious formed probable connection false

7) Holmes’ mannerism

holmes head hands shook easy smiled sank sunk breast short forehead gesture rubbed began forward clapped despair branch leaning eagerly

8) Villains in Sherlock Holmes

great doubt criminal dangerous country brain set career act failed makes gang cunning power war europe compelled sufficient traced remains

9) Smoking in Sherlock Holmes

sat pipe fire looked time cigar tobacco smoke asked sherlock corner long chair armchair smoked lit roylott smoking moran observe

10) Accents in Sherlock Holmes

don ll ve won talk thing give answered didn bit ready bad couldn wait eh minute masser wouldn isn lucky

 

 

Topic Modeling Project

~ Annie Swafford

Here are my (informed) guesses for the theme names. The data is based on me running 150 topics, 2,000 iterations, and 30 words.

Clues:paper note read letter papers word book answer letters table importance handed short written wrote received message writing pocket attention sheet write account reading von post secret picture men document

Reaction:face cried hands eyes turned instant holmes suddenly back voice looked head sprang forward words spoke lips threw moment step quick hand feet shook raised amazement staring moved burst stepped

House: room door window open light opened entered floor bed key bedroom closed lamp sound passage safe table bell steps sitting rushed inside ran pushed locked dressing lawn stair stood study

Crime: found police crime place death evidence brought case murder hopkins dead finally making tragedy body remained person arrest unfortunate attempt order hotel violence prisoner missing inspector inquiry appeared naturally weapon

Time: night morning day hour house back evening clock work late heard happened quarter room past start early breakfast returned waited hurried usual mrs left morrow arrived ten signs occurred twelve

Gun: back heard hand head stood lay long struck held turned fell deep round sight dreadful caught blood moment body cry horror strange revolver shot blow lying gave surprise dropped arms

Case: watson case point find dear clear doubt end points possibly affair person follow investigation obvious surely simple difficult clue perfectly confess theory close admit remarked undoubtedly prove suggest formed solution

Estate: house round side front road passed place long hall carriage garden high drive walked direction windows drove dark led corner pulled reached line square standing coming low past miles slowly

Travel: street found station train lord baker office waiting st cab minutes evening started yard half quiet gentleman company west made monday town detective official afternoon scotland service home reached engaged

Physical Description: face eyes black man red dark white figure hair thin light side tall heavy fashion hat appearance blue expression pale broad dressed features mouth centre middle yellow dress sharp brown