Research blog

Over on Aeon, psychologist Robert Epstein wrote a very interesting piece about how we often explain the functioning of the brain as if its a computer, retrieving memories and processing information, despite the lack of scientific grounding for that metaphor. Our brains don’t “recall” information, they “re-live” it. Information is not “stored” in neurons: our brain constantly evolved based on our experiences, and events can trigger our brains into re-activating the areas that were activated during a previous event, thus creating the impression of a “memory” for us.

This, I think, is the flip side of a discussion that I have had very often this past year about whether AI as it exists today can qualify as “intelligence” in any way. Just like “the computer” is not a very good metaphor for the human brain, the human brain is not a good metaphor for how AI algorithms work, yet this metaphor has completely taken over the way we think about AI. In fact, the whole vocabulary around AI is filled with human-intelligence metaphors: neural networks, learning, AI that explain their reasoning… and obviously “intelligence” itself. These metaphors may be useful to get a surface-level intuition about how AI algorithms work, but they are very much untrue and should not be taken as more than that: metaphors.

There’s this idea that, if we can make an artificial neural network that is as complex as the human neural network, then we should be capable of creating an actual intelligence. But that makes two hypothesis which are unsupported by evidence: that artificial neural networks are good models of human neural networks (they aren’t), and that human neural networks are solely responsible for human intelligence. That doesn’t seem to be the case either. Our brains work as part of an intricate, interwoven network of systems which all interact with each others in ways that are very hard to capture. Our neurons without our endocrine system, our immune system, etc., are a very incomplete snapshot of who we are. We can’t – and maybe won’t ever be able to – upload our brains to the cloud.

Artificial Intelligence is not Human Intelligence, and it doesn’t really make any sense to compare them in any way. Any AI algorithm has capabilities and limitations, and we can study those without falling into the trap of anthropomorphism. That’s why I don’t like it when LLMs are compared on benchmarks such as law exams, or common coding exercises. Those tests were designed for humans, to test capabilities that are often difficult to acquire for human intelligence. For an AI, it doesn’t tell us much. Importantly, it doesn’t tell us that this AI is a good lawyer, or a good software developer. Yet these are still the kind of tests used to compare LLMs today. Perhaps the most used test right now is the “Massive Multitask Language Understanding (MMLU)” test. The MMLU is composed “of multiple-choice questions from various branches of knowledge”, which were “manually collected by graduate and undergraduate students from freely available sources online”. But we aren’t using LLMs to solve multiple-choice questions from various exams and tests collected around the web. For any actual task that we have in mind for such a model, we would need to prove that the MMLU is a useful proxy to evaluate the capability of the model to perform the task.

The fact that the MMLU is collected from online sources also makes it extremely difficult to use it as a benchmark for modern LLMs, which are often trained on very opaque datasets, also collected from all around the web. The risks of contamination are huge, and the measures taken by LLM developers to mitigate this issue are often doubtful, if they are even described at all. The Gemini paper, for instance, states that they “search for and remove any evaluation data that may have been in our training corpus before using data for training”, but don’t provide any detail as to how this search was done. If they used the same method as GPT-4 (i.e. looking for exact matches on substrings of the questions), then the risk of contamination is high.

There is often a very large gap between the performances of the LLMs in benchmarks and their performances in real-world applications, and this is probably part of the reason: the benchmarks are not made for the evaluation of a computer program, but of a human brain… and those are way too different for it to work that way.

I must confess that, sometimes, I Google myself. I typically know what I’ll find: I have enough of an online presence that most of the results are related to me (and one other Adrien Foucart who competed in Judo fifteen years ago), and it’s typically a mix of my blogs, scientific papers, social media, and from time to time a post about things I’ve written or done. Finding those is the main reason that I do this exercise every once in a while. Today, however, I got a surprising result, from a website that I had never heard about: SciSpace.

SciSpace is yet another GPT-powered chatbot, aimed at scientists who want to outsource their research to a machine. You ask a question, it answers with a summary built from scientific papers, with citations to those papers so that you can read them if you want to do some work somewhere in the process. I don’t think it’s a good idea: doing those kind of summaries is how you actually gain the understanding of your field, and you’ll necessarily miss a lot of the nuance of what’s happening in the field if you just get the AI-generated “summary”. So even if it worked perfectly as advertised, I wouldn’t recommend using it. But the reason I’m writing this is that it fails pretty spectacularly at its job.

It seems that SciSpace allows you to browse questions, presumably asked by other users. Google indexed a question where, surprisingly, I appeared in the answers. I say surprisingly because the question is not quite in my field: “What are the specific cultural criticisms associated with the implementation of Panopticon in various societies?”

The beginning of the answer seems to be on-topic, although since it’s a topic I know nothing about, it could all be bullshit for all I know. But it’s around the end that I suddenly appear, with this tangent:

Lastly, Adrien Foucart and colleagues critique the Panoptic Quality metric in digital pathology, illustrating the challenges of applying panoptic principles to complex, nuanced fields [10].

This is a reference to my Scientific Reports paper “Panoptic quality should be avoided as a metric for assessing cell nuclei segmentation and classification in digital pathology”, which is probably the paper I like the most out of my publications, but which has absolutely nothing to do with the Panopticon, outside of the fact that there is “Panoptic” in the title.

And this, I think, is where GPT went wrong. Because GPT predicts tokens, which encode parts of words, “Panoptic” and “Panopticon” likely share one or several tokens in common. This alone shouldn’t be enough to trip the model, but it also happens that one of the names most associated with discussion of the Panopticon is Michel Foucault, and “Foucault” also shares at least one token with “Foucart”. This taken together probably sent SciSpace’s GPT into the wrong direction. Because, as we should all know by now, GPT has no fucking idea what it’s talking about. It’s all just a statistically likely string of tokens, with no understanding whatsoever of what’s going on.

So Foucault and the Panopticon end up mixed with Foucart and Panoptic Quality, even though these come from completely different domains. I should also note that for some reason my paper is mistakenly cited as coming from “Dental science reports” instead of “Scientific Reports”. No idea what happened there, but another clear reason that this tool is absolutely useless.

From the CEO’s LinkedIn profile, the platform is “used by more than a million researchers worldwide, including Nobel Laureates”. I highly doubt, however, that Nobel prize worthy science will be done with it anytime soon.

I recently reviewed a paper which I strongly suspect, without certainty, had a little help from ChatGPT for its research. One thing that tipped me off was that it was bit confused in the evaluation metrics: it reported, for a segmentation task, both the “F1” score and the “Dice” score, which it defined as:

$$F1 = \frac{2 \times precision \times recall}{precision + recall}$$

$$Dice = \frac{|P \cap G|}{|P| + |G|}$$

Where P & G refer to the predicted and ground truth regions.

Both of these definitions are fine, but… well, before I explain the problem, let’s see what ChatGPT (version 3.5) has to say about it.

Asking the question

User:  
What is the difference between the F1 score and the Dice Similarity Coefficient ?

ChatGPT’s answer:

The F1 score and the Dice Similarity Coefficient (also known as Dice coefficient or Dice index) are both metrics commonly used to evaluate the performance of binary classification tasks, particularly in the context of image segmentation or medical imaging. While they serve similar purposes, they have some differences in how they are calculated and what aspects of the classification performance they emphasize.

1. F1 Score:

• The F1 score is the harmonic mean of precision and recall.
• Precision measures the ratio of true positive predictions to the total number of positive predictions made by the classifier.
• Recall measures the ratio of true positive predictions to the total number of actual positive instances in the dataset.
• The F1 score balances both precision and recall, giving equal weight to both. It is calculated as: F1 = 2 * (precision * recall) / (precision + recall).
• F1 score ranges from 0 to 1, where a higher value indicates better performance.

2. Dice Similarity Coefficient:

• The Dice coefficient measures the overlap between two samples.
• It is calculated as twice the intersection of the two samples divided by the sum of the sizes of the two samples.
• In the context of binary classification tasks, one sample typically represents the predicted segmentation mask, while the other represents the ground truth mask.
• Dice coefficient ranges from 0 to 1, where 0 indicates no overlap between the predicted and ground truth masks, and 1 indicates perfect overlap.

Differences

• While both metrics are used to evaluate binary classification tasks, they emphasize different aspects of classification performance.
• F1 score emphasizes the balance between precision and recall, which is particularly useful when there is an imbalance between the classes in the dataset.
• Dice coefficient emphasizes the overlap between the predicted and ground truth masks, which is especially relevant in tasks such as image segmentation, where the spatial agreement between the predicted and true regions is crucial.
• The F1 score can penalize false positives and false negatives equally, while the Dice coefficient may be less sensitive to false negatives if the overlap between the predicted and true regions is still significant.

In summary, both metrics are valuable tools for evaluating classification performance, but the choice between them depends on the specific requirements and characteristics of the task at hand.

Dissecting the answer

ChatGPT is verbose as always, but the two definitions are mostly correct. Nothing much to say about the F1-Score, and for Dice the only real problem I have is with “measures the overlap,” as I’d rather say “is one of the measures of overlap.” The other main one being the Jaccard Index, also known as “Intersection over Union.”

The “differences” section is where I have way more problems. F1 score is balanced in terms of precision and recall, but it is not at all appropriate when there is a class imbalance (see Chapter 4 of my thesis for more!). And “the Dice coefficient may be less sensitive to false negatives if the overlap between the predicted and true regions is still significant” is nonsense.

But that’s still not the main issue. Let’s see if I can get ChatGPT to notice it.

One step further

User:  
Express both the F1 score and the Dice Similarity Coefficient in terms of True Positives, False Positive and False Positives.

ChatGPT’s answer

Sure, here are the formulas for both the F1 score and the Dice Similarity Coefficient expressed in terms of True Positives (TP), False Positives (FP), and False Negatives (FN):

• F1 Score:
  • Precision (P) = TP / (TP + FP)
  • Recall (R) = TP / (TP + FN)
  • F1 Score = 2 * (P * R) / (P + R)
• Dice Similarity Coefficient:
  • Dice = (2 * TP) / (2 * TP + FP + FN)

In both cases, TP represents the number of true positive predictions, FP represents the number of false positive predictions, and FN represents the number of false negative predictions. These formulas allow us to calculate both metrics based on the counts of these classification outcomes.

Almost there

The formulas are still correct ! But weirdly, ChatGPT didn’t exactly do as I asked. It still puts the F1 Score in terms of Precision and Recall, not in terms of TP, FP and FN. Let’s prompt it a bit further.

User:  
You put the F1 Score in terms of P and R, give the complete formula for the F1 Score based on TP, FP and FN.

ChatGPT’s answer

Apologies for the oversight. Here’s the complete formula for the F1 score expressed in terms of True Positives (TP), False Positives (FP), and False Negatives (FN):

F1 Score = (2 * TP) / (2 * TP + FP + FN)

This formula directly uses the counts of true positives, false positives, and false negatives to compute the F1 score, without needing to calculate precision and recall separately.

There we go !

So let’s put the two formulas next to each other:

$$Dice = \frac{2 \times TP}{2 \times TP + FP + FN}$$

$$F1 Score = \frac{2 \times TP}{2 \times TP + FP + FN}$$

They are the same thing ! Dice and F1 Score are two different names for the same thing. The only real difference is in when they are often used. F1 is a more common terminology in classification and detection problems, whereas Dice is more often used for segmentation problems. But they are the same.

All the talk about their differences was complete bullshit. But it would have been relatively difficult to spot without already knowing the information. Which is always the problem with trying to use ChatGPT as a source of knowledge.

Conclusions

It was not a very good paper, and whether the authors misunderstood evaluation metrics because of ChatGPT or all on their own, I don’t think their study will be published, at least in the journal they were aiming for.

But after more than a year of people trying to convince me that ChatGPT really is usually right, and useful as a source of information… I remain firmly convinced that it’s only “right” if you don’t know the topic and can’t fact-check the answer fully. In this case, the information is easily found on Google (even on Bing !). On Google, the first line I see with the same original prompt is: “Dice coefficient = F1 score,” from a Kaggle notebook.

Sure, “if I upgrade to the paid version” it may be better. The only thing that really makes ChatGPT-Plus better is the capacity to search the web. I’d rather do that directly on my own – and see the information in its context, so that I can critically assess it.