The scary thing about ChatGPT is how good it is. I asked it for a discussion on the Hebrew grammar of Genesis 1:1-3 and it was frighteningly thorough and accurate. I almost asked it to discuss its sources for what it had said but wasn’t sure I wanted to know since I’d probably end up spending all day chasing them down!
The big issue with it is that it’s already been shown to be biased on controversial topics, not by its own nature but due to the volume of passionate views on the internet and its apparent ability to be prejudiced towards “enthusiastic” positions. I’ve tested it on other things I know about, such as painting houses and dune ecology, and it is very, very good on those.
OTOH it seems remarkably ignorant of some things. For example, I asked it if Germany should restart its nuclear plants, and it only discussed environmental, safety, and public opinion aspects and had not a word about the situation regarding Ukraine and Russian oil.
Given the evidenced bias plus its apparent blindness about some things, and how volatile some issues raised here are, I wouldn’t consider it something to be quoted unless it had an insight the user hadn’t previously encountered or an elegant and/or pointed turn of phrase – and even then, anything it says should be in quote marks (I notice a bit of a deficiency in utilizing that feature) and clearly identified as to source. If quoted, it should be treated at best as a journalistic source even if – as with my question about Hebrew grammar – it provides a scholarly response.
Overall I’d say that ChatGPT material should be treated like images – meaningless unless they serve to spark useful thoughts by the people posting them – or like articles on science by popular science journalists when writing a paper for a university course, i.e. not to be mentioned but as bridges to finding actual sources.
I can’t resist: I asked ChatGPT how to use its answers in online discussions:
- Verify the information: While ChatGPT is designed to provide accurate information, it’s always a good idea to fact-check the information provided before using it in your own posts or replies. Look for additional sources to confirm the accuracy of the information.
- Use ChatGPT’s answers as a starting point: ChatGPT’s responses can provide a helpful starting point for further discussion or research. Use the information provided as a launching pad to explore the topic in more detail.
Which can be read as I said: ChatGPT isn’t a source that should be quoted, it’s a tool for drawing on like any other, and what you post should be your “own work”, not quotes!
And in terms of science and theology, its material should be treated the same as that of a very good science journalist or theology journalist trying to explain things on a popular level… and who because they are not scientists or theologians can get things very wrong.