Introduction

Among the enormous volume of tweets exchanged during the 2016 US presidential election campaign, millions were sent by alleged Russian troll accounts. Those accounts have been accused of manipulating the campaign and its outcome.
Given how dramatic and dangerous the consequences of a country influencing the democratic process of another can be, we want to get a better insight on the methods and the real effects of the trolls.
On one side, we compare their activities with the evolution of popularity of the candidates along the campaign. We will also take a look at the major events of the election and determine if they were influenced by the trolls and vice-versa.
On the other side, we will study the methods employed by the Russians to disrupt the presidential election. We will examine their vocabulary, the subjects they discuss as well as the media and people they refer to.
We also keep an eye on the strategy employed by Russian trolls in other countries, and mainly in Ukraine.

Our analysis will mainly focus on trying to provide an answer to the following questions.

  • Is there a relation between the candidates popularities and the activities of the trolls?
  • Did the trolls influence the major events of the campaign? Is it the other way around?
  • Which subjects are discussed by the trolls?
  • Which media do they tend to talk about and link in their posts?
  • Do they tend to show direct support or hatred for specific people?
  • Did the strategy of the trolls change over time?

Dataset

The main dataset we are going to use for this project is the Russian and Iranian Troll dataset from Twitter. It contains over 10 million tweets of accounts associated with IRA and Iranian trolls. It gives information about the account itself (Twitter handle, number of followers and following, account preferences...), and about the individual tweet (content, date and time of publication, type of post, number of retweets...).

Regarding the popularities of the candidates, we have identified an important amount of data source on realclearpolitics.com.

We also took a look at the events after the campaign, with for example the evolution of the approval rate of Donald Trump proposed by FiveFirtyEight.

Finally, for the timeline of the campaign and its major events, Wikipedia has a lot of information.

NB : Not all the tweets collected are trolls per se. The tweets originate from alleged troll accounts that were identified by Twitter. However, in order to not be spotted immediately as fake foreign accounts they also tried to pose as any regular American or Russian account. Therefore the dataset also contains plenty of seemingly innocent tweets.


Global Temporal Analysis

In order to assess the volumetry of tweets sent by IRA's identified troll farm we would like to display the overall frequency distribution of the tweets. In addition, we would like to observe the evolution of the number of troll accounts with time. We might end up with an interesting trend that could be explained/supported by events in the actuality at the time being.

We notice that the first recorded tweets from troll accounts go back to 9th of May 2009: coincidentally, Donald Trump's first tweet dates from the 4th of May 2009.

We identified 5 main periods on the timeline. Here we give a first insight on each period and its meaning. More detailed analysis will be conducted for each period later on.

  • Period 1 (US 2012 Elections) : In yellow we can observe a first increasing trend that is shaped like a hill. It's located between February 2012 and June 2013 with a noticeable spike in October 2012, the month right before US 2012 Elections. Taking a look at the graph showing the evolution of the number of troll accounts with time we can argue that for such a small amount of them during period 1 they had to be quite active for that campaign.

  • Period 2 (Hiring Time) : Starting from mid 2013 and mostly at the beginning of 2014 we can notice a sudden increase in both the number of accounts and the troll feed activity, which are obviously correlated. IRA's page on Wikipedia underlines the job proposal made in August 2013 :

    " Internet operators wanted !"

    Wired sheds light on the early practices of IRA. Actually the initial goal of the agency was to interfere on several events of the news (elections, war in Ukraine, terrorism... massive opportunities!).

  • Period 3 (Donbass' War, Full Swing) : This time window corresponds to a massive tweet campaign to support pro-Russian factions in Donbass war in Ukraine.

  • Period 4 (US 2016 Elections) : This period of high activity embeds both Ukraine's war (2014-present, but its intensity reached a peak in mid-2014) and US 2016 presidential elections. After, the troll fever seems to drop monotically until June 2016 before growing up again right before the final elections.

  • Period 5 (Ukraine's war & Presidential Instability) : The activity of troll accounts did not decrease right after the elections. The contestations of the election results, the presidential transition, etc. all make great troll topics. We have to keep in mind that until the end of January 2017, Trump's election was not confirmed yet by the congress ( wikipedia/2016 US elections ). Trolls stop after 100 months, at the beginning of the academic year 2017-2018 (end of October 2017).

Textual analysis

Before doing further analysis let us first see what the Tweets actually contain.
That is, let us take a look at the

  • Languages : What are the languages used by the trolls ?
  • Hashtags : Which hashtags do they use ?
  • Words : Which are the favourite words of the trolls ?
  • Sentiment analysis : What are their opinions on controversial subjects ?
  • What are the mains differences between Russian and English trolls ?

Languages

As we can see, the vast majority of trolls are Russian and English.
The und stands for 'undefined'. This can be explained by the facts that Tweets are written in a poor grammatical structure, making them difficult to be understood by a computer.
Note : for the rest of the analysis, we will use the english tweets and a translated subset of the russian ones. This subset corresponds to the 15.000 most retweeted ones.

Hashtags

Another important aspect of the tweets are the hashtags. They represent a lot of information as they can tell about the subject of the tweet, the opinion of the tweeter, ...
Let us compare the most popular hashtags between Russian and English ones.
As we can see the most used hashtags are those about news, sports and politics. This makes sense as these 3 topics are common subject of dicsussion between all tweeter's users, trolls included.
A quick google search about these tweets teaches us that
  • ВСУ is the "Armed Forces of Ukraine"
  • ДНР is Donetsk People's Republic which is a proto-state of Ukraine
  • ПозорWADA talks about the cancellation of the Russian paralympic games
  • АнгелыВсердце refers to the events of the killed children of Donbass
  • ЛНР means Luhansk People's Republic which is another proto-state of Ukraine
We can clearly see the difference of discussed topics between Russian and English trolls.
Now is there a difference between trolls and normal users ?
By consulting webistes such as theroot, WebBureau, fin24 and the independent, we can draw the following tweet's usage
Top 10 most used hashtags in 2014, 2015,2016 and ever
2014 2015 2016 Ever
#BringBackOurGirls #JeSuisParis #Rio2016 #BlackLivesMatter
#TweetLikeJadenSmith #BlackLivesMatter #Election2016 #CupforBen
#BreakTheInternet #MarriageEquality #PokemonGo #brexit
#LifetimeMoviesBeLike #LoveWins #Euro2016 #EdBallsDay
#DonLemonReporting #IStandWithAhmed #Oscars #FollowFriday
Note that we have chosen the years 2014,2015 and 2016 here because they correspond to the peak of trolls' activity. One can already observe major differences between the trolls and normal people' discussions. Trolls tend to talk more about global subjects while others are highly influenced by local events.

Words

Now that we have analysed the hashtags, let us focus on the words themselves. This allows us to see what are the typical words used by trolls. Note that in order to stay the more coherent possible we've stemmed the words and did not include the stopwords in the list.
Without any surprise the most frequent word is Trump.
Now comparing those with the Russian tweets
Once again, we can clearly see the difference of discussed topics. Russian trolls are naturally more focus on local subjects such as Ukraine or the genocide of Donbass with words such as children or memories.
Nevertheless the attentive reader may notice the presence of the word american at position 18.

Are those words very different from those of a typical English tweeter user ?
Long story short : kinda.
The biggest difference between is obviously the total absence of the word Trump for the normal users ! This indicates well how Trump is a particularly specific topic for the English trolls.
Furthemore, except for the word love, one could argue that the words used by the English trolls seem to be more negative that those used by the rest of the world. This is illustrated by words such as kill, break, never and shoot for the trolls, where other prefer words such as good, lol, great.

Sentiment analysis

The final part of this textual analysis consists of classifying the relevant tweets as positive or negative. Such a process can be performed via a sentiment analysis.
Once again, we have separated the analysis of the English and Russian tweets. It must be noted that two parallel analysis were made. A quite naive one inspired by Jeffrey Breen and a more complex one based on a MIT tool.

For the English tweets, we focused ourselves on the tweets talking about respectively Trump, Obama and Hillary.
As we can see, the negative scores of Trump and Hillary are close enough while Obama has the lowest one. As for the positive scores, Trump is the big winner here against a Hillary that is quite behind.

As for the Russian tweets, we've decided to dig deeper into the Ukrainian conflict illustrated by the words and hashtags analysis. As a comparison, we've also selected Russia, another highly debated subjects among the Russian trolls.
Here the results are clearly visible, not only the negativity towards Ukraine is much more pronounced than towards Russia, but the opposite goes also for the positivity; the tweets are much more positive when talking about Russia.


Detailed Temporal Analysis

Time Window 1 (Vladimir Putin and US 2012 Elections)

Small interactions with US 2012 elections and easy to spot intra-Russia tweets about the elections.

As previously highlighted a peak of activity is worth mentioning 1-2 months before final US 2012 presidentials (light blue) Furthermore one can notice a serie of saddle points/plateau (green) around May 2012 : actually looking at the most retweeted tweets for period 1 it comes to light that Vladimir Putin has been put forward by troll accounts at that moment. This coincides with the inauguration of his third presidential term, which was accompanied by protests in the streets of Moscow.

Interesting Tweets

Among the 10 most retweeted directly-politically-related-tweets for the concerned time period, 3 really did catch our attention.

3 hatred tweets and their meaning in english
Date RU : text EN : text
2013-02-04 (after US 2012 elections) Давайте перенесемся в 1991 год. Вспомним все надежды, на демократию, Ельцина, радость от краха КПСС, и посмотрим на нашу действительность. Let's fast forward to 1991. Recall all hopes for democracy, Yeltsin, the joy of the collapse of the CPSU, and look at our reality.
2012-03-03 Когда вы будете вводить демократию с помощью оружия? When will you introduce democracy with weapons?
2013-05-11 (after US 2012 elections) Первые демократические #выборы в #Пакистан'е прошли на фоне взрывов и стрельбы. Власти отмечают высокую явку. Obama fell off his bike and suffered responsibility for the & lt; & lt; terrorist attack & gt; & gt; a terrorist group from & lt; & lt; Al-Qaida;
Highlighted tweets above are quite expressive... and representative of the very start of the trolling wave, trollers used to employ gross/violent language and send content of hate sometimes. It's not because the elections are over that trollers stopped their inference duty : for instance, for the third tweet of the above table they want to put discredit on Obama with respect to a terrorism topic.

Time Window 2 (Hiring Time) and Time Window 3 (Donbass' war)

Recall :

  • Fact 1 : We observe increasing troll activity immediately time-correlated to the start of the well known armed conflict in Ukraine.
    The semantic analysis performed above already showed the overall support tweets from IRA tried to send through.

  • Fact 2 : IRA launched a campaign during period 3 in order to recruit more and more people (hiring campaign of August 2013)

  • As the Wikipedia page for war in Donbass tells us, Ukrainian armed conflict started around March-April 2014, then with some latency trollers began their job apparently.

    Prior to those events a calm period had been observed, Ukrain's troubles seemed to trigger the trollers again. During the next time window we've spotted three major events (by this we mean events that were massively relayed by medias) for which IRA trollers did react.

    Among them we can notice the shootdown of the russian war aircraft :
    Generally trolls spread fake news to defend the Russian regime.

    Time Window 4 (US 2016 Elections)

    Let's now focus on the period of time corresponding to the whole presidential campaign (+ some time before as a flourishing speculative period)
    We can see that peaks of the second graph do not always match with the most prominent peaks within the overall frequency distribution of troll tweets (presence of other topics : like Crimean conflict). Events of the first importance are immediately visible through the second graph ; that is the graph obtained after filtering out non directly politically related tweets. This allows the reader to better see the moves of IRA with the time.

    • 1 : In green, the observable peak of US political troll tweets is directly correlated to Donald J. Trump announcement about his candidacy to the US 2016 presidential elections. Trolls made a lot of tweets about that fact to best spread the word. (6th of June 2015)

    • 2 : In light blue, was held the second republican debate, Donald J. Trump did attend this event. (16th of Sep 2015)

    • 3 : In salmon, Hillary Clinton's mail gates. At the time of the peak FBI's former director, Comey (who will be fired several months after as we will see), testifies about the extremely careless attitude of Hillary towards her mails while being state's secretaress for president Obama. Those revealings were a really good opportunity for trollers to lead a front attack against Clinton.

    • 4 : In purple, took place the first presidential debate between Donald J. Trump and Hillary Clinton : (26 of september 2016, after the primaries)
      Event followed by many people and for which the outcome (win the debate/lose the debate) is sometimes crucial in a campaign.

    What's more ?

    We have succeeded in spotting some peaks immediately related to campaign's events !

    In order to get a quantitative effect of the trolls tweets we will display the aggregated sentimental scores per week and per opponent.
    We have aggregated the number of positive/negative tweets referring to Trump (resp. Clinton) (following our sentimental analysis in the previous section) per week during the period 4 (whole presidential campaign). Then we computed favorability deltas that were just the difference between the positive and the negative scores (tweets counts) of a candidate during a week. Let's review the evolution of the favorability deltas (using more advanced sentimental analysis)
    Here below are given summary statistics about these weekly sentimental deltas !

    Summary of Sentimental deltas
    TRUMP : summary statistics of sentimental deltas CLINTON : summary statistics of sentimental deltas
    min -127 -54
    mean 1.52 -3.15
    max 362 28
    We actually find out that Trump is slightly supported by the troll tweets with respect to Hillary which is, in average, disapproved. Of course, one has to remain careful while reading those stats since the undertaken sentimental analysis isn't a pure science yet : quite heuristic! For instance, the trends are too hard to catch and we cannot really tell how did the strategy of troll accounts vary over time. Note that the median weekly scores is almost 0. It indicates that there are plenty of weeks for which no hatred support is remarkable.

    We tried to put side to side intentions of votes for Trump (resp. Hillary) with the volume of politically related tweets to see if there were easy to catch correlations but we did not bring to light any apparent results. A lot of different social media sources must be taken into account, it is really hard to get significant results only based on the given datasets. Most of the interesting statistics about social influence / popularity scores ... etc are chargeable (you must pay for them), also we should have had access to studies (for instance the studies used by AllCott-US2016-fakeNews) to get an insight on how are people affected by the presence of the well established troll tweets. However the existence of trolls is undeniable (examples are to follow below) but their spread to american citizens (or more globally, for all tweets' topics, human beings) is really tough.

    Most of the interesing tweets (with respect to the content) are among the most retweeted ones. Having a closer look to the 1000 most RT tweets during the period 4 (preceeding the final elections), we managed to spot an user profile that produced up to 56.7% of those 1000 tweets!

    Here is a sample of his achievements :
    3 hatred tweets and their original content in english
    Date EN Tweet Recount HashTags
    2016-10-11 OMG, this new Anti-Hillary ad is brilliant!👌 It's fantastic!!!!!! Spread it far & wide! 10756 []
    2016-10-18 RT the hell out of it: Dem party operatives: 'We've been bussing people in.. for 50 yrs and we're not going to stop now' #EvangelicalTrump 6772 [EvangelicalTrump]
    2016-10-20 🚨DISGUSTING Watch: Hillary laughing when Trump said gays get thrown off buildings in Muslim counties‼️‼️ #debatenight #trumpwon #debate 3762 [debatenight, trumpwon, debate]
    For instance, the following tweets are highly against democrats and more precisely Hillary. We are not surprised by the use of the hashtag #trumpwon : this does not contradict the thesis for which the outcome of the debate does matter.

    Time Window 5 (Ukraine's war & Presidential Instability)

    In this time window we did observe two main events that pushed massively troll accounts to their keyboards.
    We only focused on the political tweets since the peaks of activity are much more visible that way.

    • 1 : In green, the peak corresponds to Times' revealings in July 2017. These revelations pushed the US into a big national debate and a lot of political ruction.

    • 2 : In light blue, the victory of Trump at US 2016 elections, welcommed by IRA's trollers.

    Interesting Tweets

    Among the 10 most retweeted directly-politically-related-tweets for the concerned time period, 3 really did catch our attention.

    (examples) 3 hatred tweets and their meaning in english
    EN : text
    JULIAN ASSANGE: I investigated both presidential candidates — Hillary was the only one with corrupt ties to Russia https://t.co/MVVPkLORTbсть.
    President Trump calling Jim Acosta/CNN fake news is officially the best thing I've seen this week. Cry baby Jim, cry!
    So much evidence that Comey covered up civil rights abuses. Certainly a bigger story than that Russian #fakenews https://t.co/OqKw5dqU19
    Those tweets are reactions about that "scandale" earlier in 2017 about Trump's collaboration with Russia and his false statements on social medias.

    Looking at the global timeline of events during the campaign : around July 2017 8th, Times revealed (accused) Trump had been negociating with russian contacts in Trump's tower. It was proven 6 months later to be true but didn't fail to attract a lot of mediatic buzz. As the tweet_texts mention it there was also Comey's gate (FBI former director) ! Trolls here are just using the most basic technique : bad faith !

    The most retweeted tweets speak for themselves :

    • Comey is a liar and shouldn't investigate about Trump's legitimacy.

    • Hillary is the only one corrupted w/ russians.

    • To Trump, journalist Acosta is "fake news" (it launched a new trend by the way)



    Retweets

    Retweets are one of the most important functionalities of Twitter. They often lead to a snowball effect: a post that receives attention gets retweeted several times, which gives it more visibility, thus more attention... Retweets create retweets! It is a really convenient and fast way to propagate a message.

    Retweeted by the trolls

    A quick glance at the messages written by the trolls that got the most retweets shows us that Russian is the main language for those. It has twice the count of English.
    Let us now take a look at the exposed accounts (trolls or not) that were the most retweeted by the trolls. Regarding trolls, we only take into account those that are not anonymized, as they are by definition more popular, and they can offer us more information.

    We can see that the exposed trolls present in our database do not represent the majority of the most retweeted handles, only 6 of them are in the top 20 (one of which is anonymized). Besides those, two accounts have been deleted and their handles are unretrievable. All the accounts in this top 20 appear to mainly write in Russian (shown in red on the graph). This also holds true for the troll accounts: even though some of them picked English as account language (shown in blue on the graph), they all tweeted mainly in Russian.

    Most of the most retweeted accounts are news agencies, several of those are owned by the Russian government. It is also interesting to note that almost all of them experienced major changes in 2013-2014. Here are the most notable ones:
    • rianru : Refers to RIA Novosti (Russian news & Information Agency, "Novosti" means "news"), the previous official international news agency of Russia. It has been replaced in 2014 by Rossia Segodnia, but the name RIA is still used.
    • GazetaRu : Gazeta.ru is one of the most popular Russian online newspapers. Was part of a major companies fusion in 2013, under the name Afisha.Rambler.SUP, later renamed Rambler&co in 2014.
    • RT_russian : Previously named Russia Today, now simply known as RT (sheer coincidence?). This news channel was created by RIA Novosti, and is globally considered as a propaganda outlet. New centre in 2013, RT UK launched in 2014.
    • vesti_news : A news channel belonging to VGTRK, a media company owned by the Russian government. Experienced a brand refresh in 2014.
    • lentaruofficial : Lenta.ru is another popular online newspaper which has been restructured in 2014 and has since be accused of serving propaganda purposes.
    • tass_agency : Also a government owned news agency, of which the name changed in 2014.
    • lifenews_ru : News website, pro-Kremlin, and allegedly spreading fake news. TV channel launched 2013.
    • riafanru : This is the most retweeted troll account, with a counter exceeding 70 000 (almost three times the counter of the second most retweeted troll). The corresponding website (Federal News Agency) seems to be focused on Ukraine, Syria, Russia, and fearmongering. It was founded in 2014 and is still running in 2018. (Account active between 2015-2017.)
    Clearly, Russia decided to implement some serious changes in its media landscape around 2013-2014. This period corresponds to the explosion of the number of troll accounts.

    Retweeters

    Now that we have seen the most retweeted accounts, let us consider the main "retweeters", focusing again on the unanomyzed accounts. Surely, since Russian accounts are the most retweeted, Russian accounts should also be responsible for most of the retweets?
    Surprisingly, the vast majority of the main retweeters have their account language set to English, and actually tweet in English. Several of those are classified as hashtagers, accounts that play games by replying to popular hashtags and using those to share their political stance.

    Trolls retweet network

    It is now time to visualize the links between the troll accounts that are not anonymized. As above, accounts with English language settings are in blue, while Russians are in red. The size of a node is proportional to its degree. The color of a link indicates its importance: red links correspond to more interactions between two accounts.

    There are clearly two main clusters, divided by language. Again, some accounts with English settings are actually tweeting in Russian. All the accounts in the "Novosti" (recall that it means "news") cluster are in this position.
    The two big groups are however connected! Those are the links between them:

    EN
    RU
    WarfareWW zubovnik
    WarfareWW coldwar20_ru
    WarfareWW ComradZampolit
    WarfareWW MatEvidenceRU
    Pamela_Moore13 russilanrogov
    TEN_GOP russilanrogov
    EN
    RU
    MatEvidence zubovnik
    MatEvidence coldwar20_ru
    MatEvidence ComradZampolit
    MatEvidence MatEvidenceRU
    MatEvidence NovostiDamask
    Four English speaking and five Russian speaking accounts make the bridge between the clusters. Two of the American accounts are right trolls, while the two others (which show a lot of similarities) are classified as newsfeed.
    russilanrogov is the only Russian connexion of Pamela_Moore13 and TEN_GOP. It also seems to have right troll tendencies. Although it writes mostly in Russian, several of its tweets are in English or even in French, and are very focused on politics.
    Interestingly, the relations always appear to go in the same direction: the Russian accounts retweet the English speaking ones, and not the other way around.
    Following an article by The Washington Post, it has been observed that several close relatives of Donald Trump have engaged with some of the accounts mentionned in this table, one of which being on the RU side.

    In brief

    Regarding retweets, the IRA trolls are split in two main groups: Russian speaking and English speaking.
    The Russian trolls tend to interact much more between each other, as we can see both on the bar charts and on the network graph. They also happen to sometimes use posts from American trolls, while the contrary does not appear to be true. Furthermore, they seem to focus their activity on a handful of very popular media, the majority of which is directly linked to the government.
    English speaking trolls, on the other hand, seem to use retweets very actively and to diversify their sources. The graph below shows that they also are much more successful in spreading their messages through retweets. The posts written in English by the trolls got more than twice the number of retweets as the posts in Russian.

    URLs

    Another way to understand better what the trolls are talking about is to analyze the urls present in their tweets. There are more than 4 millions of them, some tweets contain more than one.

    Domains

    Let us first get an overview of the domains that are the most used, and compare them based on the languaged of the tweet in which they appear. Clearly, the popularity of the domains are mostly determined by Russian tweets. We can already see that some websites are used a lot by both communities. bit.ly, ift.tt, goo.gl and j.mp are all url-shorteners ; ift.tt also provides social media management services, like dlvr.it. twitter.com and youtu.be are of course very popular social media, and also propose shortened versions of their urls. Those short urls are very useful on Twitter, due to the character limit, which explains the popularity of those domains.
    Splitting the domains between the two languages makes it easier to compare and analyze them. The most popular domains present in tweets in English are rather unsurprising: Twitter, many url-shorteners and popular video media like Youtube and Vine. The only weird domain is LoseFatTips.pw. From what we can find on the Wayback Machine, it seems to simply be a shady website which is now terminated, that used to redirect visitors to Asian websites. Its presence in the top 10 is probably due to one account spamming it constantly. The popular domains found in Russian tweets are a bit more informative than the American ones. The usual url-shorteners and social media management tools are present, but they compete with several news websites. Several of those are recognizable: they correspond to some of the most retweeted Twitter handles: riafan.ru, gazeta.ru, russian.rt.com, nevnov.ru, and vesti.ru.

    URLs

    Going a bit deeper, we observe that some URLs are actually repeated a few hundred times. They are listed in the table below, according to the language of the associated tweets. Although several links are dead, some of them can be retrieved thanks to the Wayback Machine. Russian sites can be translated with an online tool such as Lexilogos to get an idea of their content.

    RU
    URL
    Count
    [REMOVED] Kiev-news.com 2867
    Negotiations between Ukraine and Russia are "not easy" 2800
    [REMOVED] Kiev-news.com 2178
    Nigeria has an agreement with terrorists "Boko Haram" 1284
    Senegal first on the planet defeated Ebola 1170
    [REMOVED] IFTTT picture 1095
    In the Donbass, additional identification points for the dead in the ATO zone are open 793
    tvrain.ru 2867
    Tajikistan unblocked access to social networks 782
    Who is to blame for the Donetsk air crash? 779
    EN
    URL
    Count
    1063 ATL Radio 4357
    U.S. Freedom Army 3662
    The Telegraph 2369
    covfefe.bz (redirects to hedgeaccordingly.com) 2035
    twibble.io 1719
    theunder.us 1255
    [REMOVED] Payday-Loans-24.com 1203
    WKMT-DB Dagr8fm 835
    [REMOVED] Crisis in nuclear power engineering of Ukraine may cause new Chernobyl 799
    StopBeefinRadio 730
    On the Russian side, the majority of the URLs lead to articles about foreign politics. Ukraine seems to be involved the most. This is in line with what we have already seen: Russian trolls tend to focus on (fake?) news, and they like to target Ukraine.

    The English speaking side however is more diverse. There are some political (fake) news websites: U.S. Freedom Army, covfefe.bz, or even The Telegraph, a British newspaper which has a notable conservative stance and which has been accused of spreading pro-Russian propaganda. The most intriguing part of this ranking is the presence of four webradios. They do not appear related, and only one of them offers some kind of news reports. Their purpose is unclear.

    In brief

    The URLs are coherent with what we learned previously. Besides the usual popular websites, we can clearly distinguish the behavior of the trolls depending on their language. Russians focus on spreading news reports, while Americans tend to diversify their activities and to look more like casual users.

    Conclusion

    To conclude, what our analysis pointed out is the fact that there are many things a creative mind could say while performing classical data analysis on IRA's tweets dataset. As previously mentionned, we sometimes faced a lack of data. Mostly data about the real effects of an (over)exposure to fake news/trolls on the opinion of people. Such information sometimes exist but are either incomplete or not free. After understanding the impossibility of completely answering all of our initial questions, we started to think differently. We managed to discover as many pieces of evidence as possible, traces of possible interferences of IRA's trolls, both on American and Ukrainian politics.

    At the end of the day we would like to underline the correct identification of the troll accounts. Our analysis showed so many correlations and confounding events that it clearly appears those accounts were willing to spread misinformation through the Internet.

    Here are, for each analysis, the lessons we can learn:
    • The temporal analysis showed that the volume of activity of the trollers matches very well mediatic (political) events ; especially regarding US 2016 presidential elections and the Crimean armed conflict (Donbass' war).
    • Textual analysis shed light on the fact that trolls tend to use a vocabulary that is, in general, more negative. Their favourite word is Trump !
    • Retweets and URLs showed that Russian speaking accounts interact more between each other and focus on spreading messages from their government's news media. Meanwhile, English speaking accounts look more casual, and manage to get retweeted more frequently.