We have all seen those incendiary or divisive posts in our feeds, often peppered with icons used as bullet points (
, 
, 
). They do not pass the smell test. They seem fake.
The account owners are often ‘retired executives’, or ‘former enforcements/militaries’. How many of you have received connection requests from ‘recruiters’ who only have a few connections and ask for your WhatsApp number?
X (Twitter) made headlines when it disclosed that many popular political accounts were based in India and Eastern Europe. Untrue content and fake accounts abound nowadays, on all media platforms. They are used to either spread disinformation and division (which nefarious actors are behind?) or to generate clickbait by leveraging human emotions for pennies on the dollar.
The SQA/CFA conference yesterday (a tip of the hat to the organizers for the great academic presentations) highlighted an interesting research paper. The academics reviewed GenAI in trading feeds, comparing two trading news platforms. The analysis demonstrated that artificially-generated content leads to market volatility, as well as divisiveness. It also shows that, while ChatGPT is becoming ubiquitous, efficient detectors of GenAI-generated content are also available, regardless of the LLM provider. The academics used GPTZero, a highly advanced tool in that regard.
Below is a small experiment, with no scientific value, for illustration. A GPTZero analysis was deployed on one of those suspicious posts, randomly selected in my feed.
The content of the post refers to the recent ICE incident in Minneapolis, currently making headlines. Regardless of the opinion one may have on the topic, it is self-evident that the post is biased. It is not made to communicate, but to aggressively support one side of the equation. The full post is here. A partial extract is below:
And the result is:
This post was entirely generated by AI. Moreover, the author regularly reposts other GenAI-generated posts, so it is not an isolated accident.
Looking at the author’s profile, we can see that none of his employers has a company webpage. This is also suspicious. Any firm nowadays knows the importance of presenting well online. And if he is a good salesperson, as his career indicates, he would know the benefit of creating proper companies.
The post was created by GenAI and posted by an individual whose profile is also suspicious, which says a lot about the true intent of the message. At the very least, it should be taken with a grain of salt.
This was not a scientific study of effectiveness, but an illustrative case to highlight that we can efficiently detect fake content in feeds.
A few questions follow:
The tool is easy to deploy. It is free. It does not even need a registration (initially). Every rational user can (and should) verify his/her feed to avoid falling prey to people with bad intentions.
Human citizens, who communicate through and benefit from this online medium, can report such content. It is the form of self-policing that some media owners advocate.
Alternatively, the platform, its content moderators, and its management could systematize the deployment of such tools to control the flow of disinformation.
Unfortunately, the same academic research paper demonstrates that platforms have different strategies regarding original vs fake content. Some websites strategize true human-created content, more enriching and attracting premium usage fees. Some platforms prefer divisive flow and clickbait.
Whether platforms deploy those truth-sayers will also separate the wheat from the chaff, highlighting the platforms with true professionalism from the floggers.
PS: This content was written by a human. Please test it.