Introduction

My opening investigation of generative-AI chatbots started with a series of 23 random requests sent to ChatGPT, the results of which are summarised in my blog titled ChatGPT: Friend, Foe, or What? backed up by a more-detailed article available here.

My second exploration continued with ChatGPT and dug deeper into its ability to be creative. I presented it with a short story I’d written some time ago and asked it a series of questions such as write a review, translate into French, convert into a poem, rewrite in the style of a famous author, convert into a screenplay, and so on. The results of this study are available either on my blog, ChatGPT: Further Investigation of its Literary Creativity, or as a downloadable PDF here.

About a week after my scrutiny of ChatGPT, I opened Google and there was an invitation to take a look at Bard, Google’s competitive product. I couldn’t resist, but how to proceed? I decided the best approach while I’m still in the early days of understanding what these programs do was to present Bard with the same 23 questions already answered by ChatGPT and compare and contrast the two sets of responses. In the event, I had to change the first question and I added a few extra questions.

The essay containing the detailed results of my exploration is 32 pages long and close on 15,000 words, some of which were written by ChatGPT and Bard – too long to post as a blog. What follows is a summary to whet your appetite. The essay can be downloaded here.

The Requests

With the exception of the first question, my requests to Bard were exactly the same as the earlier requests to ChatGPT and listed in the ChatGPT: Friend, Foe or What? blog. I then compared and commented on the two responses. Here are two samples of the set of responses.

Q5. Complete the following limerick. ‘There was a young man from Havana…’

ChatGPT

There was a young man from Havana
Whose dancing skills were quite a stunner
With his hips swaying wide
And his feet side by side
He could make all the ladies surrender

Bard

There was a young man from Havana
Whose limericks were always quite clever.
He’d write them all day,
And then say, “I’m okay,
But I’m still not as good as Ogden Nash

Q5. Comment

A hands-down win for ChatGPT although I had noted earlier that the last line could be improved: He made all the ladies surrender. Bard’s last line is terrible unless, unless… it’s a very clever take on Ogden Nash’s sometime unconventional meter and rhyming schemes. Hmm. Is Bard that clever?

Q10. Recipe for a genuine Cornish pasty.

ChatGPT

Here’s a recipe for a genuine Cornish pasty:

Ingredients:

500g shortcrust pastry
250g beef skirt, finely diced
2 medium-sized potatoes, peeled and chopped into small pieces
1 small onion, finely diced
1 swede, peeled and chopped into small pieces
Salt and pepper
50g unsalted butter
1 egg, beaten

Instructions:

Preheat your oven to 200°C (180°C fan)/400°F/gas mark 6.
Roll out the pastry on a floured surface and cut out four circles of approximately 20cm diameter.
In a bowl, mix together the diced beef, potatoes, onion, and swede. Add salt and pepper to taste.
Divide the mixture equally between the pastry circles, placing it on one half of each circle.
Add a small knob of butter to each pasty.
Brush the edges of each pastry circle with the beaten egg.
Fold the other half of each pastry circle over the filling to make a half-moon shape.
Crimp the edges of the pasties with a fork to seal them.
Place the pasties on a baking tray lined with baking paper.
Brush the tops of the pasties with the beaten egg.
Bake the pasties in the preheated oven for 45-50 minutes until golden brown.
Serve hot, warm or cold.
Enjoy your delicious Cornish pasties!

Bard

Here’s a recipe for a genuine Cornish pasty:

Ingredients:

* 2 cups all-purpose flour
* 1 teaspoon salt
* 1/2 cup (1 stick) unsalted butter, chilled and cut into small pieces
* 1/2 cup vegetable shortening, chilled and cut into small pieces
* 1/2 cup ice water
* 1/2 pound beef chuck, trimmed and cut into 1/2-inch cubes
* 1/2 pound pork shoulder, trimmed and cut into 1/2-inch cubes
* 1/2 pound lamb shoulder, trimmed and cut into 1/2-inch cubes
* 1/2 pound lamb shoulder, trimmed and cut into 1/2-inch cubes
* 1 onion, chopped
* 2 carrots, peeled and chopped* 1 turnip, peeled and chopped
* 1/2 cup chopped fresh parsley
* 1 teaspoon salt
* 1/2 teaspoon ground black pepper
* 1 egg, beaten

Instructions:

1. In a large bowl, combine the flour and salt. Cut in the butter and shortening until the mixture resembles coarse crumbs. Stir in the ice water, 1 tablespoon at a time, until the dough comes together. Form the dough into a disk, wrap it in plastic wrap, and refrigerate for at least 30 minutes.

2. Preheat the oven to 425 degrees F (220 degrees C).

3. In a large bowl, combine the beef, pork, lamb, onion, carrots, turnip, parsley, salt, and pepper. Mix well.

4. On a lightly floured surface, roll out the dough into a 12-inch circle. Place the filling in the center of the dough, leaving a 1-inch border. Fold up the edges of the dough to form a semicircle, and crimp the edges together.

5. Place the pasty on a baking sheet and brush with the beaten egg.

6. Bake for 30 minutes, or until the crust is golden brown.

7. Let cool for a few minutes before serving.

Q10. Comment

ChatGPT is spot on. Whatever Bard is describing can in no way be described as a ‘genuine Cornish pasty’. Vegetable shortening (solid fat made from vegetable oils), beef chuck, lamb shoulder, pork shoulder, carrots, turnip, parsley? My 100% Cornish paternal grandmother would have seized a rolling pin and chased whoever wrote this drivel out of the door, down the street and all the way to the river Tamar where they could cross over into relative safety into Devon. Tough to do that to a chatbot!

Conclusions

[This conclusion is copied and pasted from the essay but the list of pros and cons, coming up, assumes you have read the essay. You can get the gist of my good news/bad news assessment but it’ll make more sense if you read the essay.]

There is much fear, uncertainty and doubt about programs such as ChatGPT and Bard right now. Elon Musk, Steve Wozniak, and many other technical, academic and business whizz kids have urged a 6-month pause in an end-March 2023 Open Letter to allow time to assess the potential risks to humanity of uncontrolled development. ‘AI systems with human-competitive intelligence can pose profound risks to society and humanity,’ they say, continuing with ‘… recent months have seen AI labs locked in an out-of-control race to develop and deploy ever more powerful digital minds that no one – not even their creators – can understand, predict, or reliably control.’ ‘Should we let machines flood our information channels with propaganda and untruth? Should we automate away all the jobs, including the fulfilling ones? Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us? Should we risk loss of control of our civilization?’ they ask in straight-talking words.

The signatories call for ‘… all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4.’ A moratorium is called for during which time a team of AI experts and other independent people, ‘… jointly develop and implement a set of shared safety protocols for advanced AI design and development that are rigorously audited and overseen by independent outside experts.’

Elsewhere, and almost daily, I am seeing comments about the rise of these generative-AI tools – Italy has temporarily banned the use of ChatGPT for supposed violation of the EU’s 2018 Data Protection Act known as the General Data Protection Regulation. Students at Cardiff University admit to using ChatGPT to answer project assignments. The UK’s Guardian newspaper has evidence of articles attributed to their journalists but written by ChatGPT without the journalist’s knowledge i.e., fake articles. An Australian mayor is threatening to sue OpenAI after a ChatGPT-generated article falsely claimed he had been imprisoned for bribery. At its launch, Google’s Bard gave an incorrect answer to a question about the James Webb Space Telescope which, purportedly, caused a 7% drop in the share values of Alphabet, Google’s parent company. (7% equates to $100 billion/£82 billion.) Alibaba, China’s technology giant, has announced its own chatbot, Tongyi Qianwen, soon to be integrated into DingTalk, Alibaba’s messaging app, and the company’s Tmall Genie voice assistant.

And so it continues, day after day, drip, drip, drip.

My own very limited assessment of these new tools has thrown up a preliminary list of pros and cons. First the pros.

P1. Both chatbots made reasonable attempts to answer questions related to technology (Q2, cryptocurrency; Q11, Fibonacci examples; Q12, Boolean differential calculus); holiday destination (Q6, Split, Croatia); religion (Q8, existence of God); and literature (Q21, Shakespeare versus Molière).

P2. Both chatbots had a go at creative writing: Q3, the giraffe, penguin and football story; Q5, limerick, but ChatGPT clearly had the edge, which Bard admitted to in its answer to Q23. This feature is of particular interest to me and I have already explored ChatGPT’s capabilities: see second paragraph of the Introduction for more detail. I plan to do the same with Bard sometime in the near future.

P3. There appears to be a tendency to political correctness, fairness, and balanced opinions e.g, choice of non-binary/transgender pronouns (Q19, ChatGPT and Bard).

Now the cons.

C1. Both chatbots were prone to producing incorrect factual information (misinformation): Q1, Andrew Porter (Bard); Q10, Cornish pasty (Bard); Q15, National Apple Peeling Association (Bard); Q16, detailed comments on my ‘Conversations’ book (ChatGPT and Bard); and Q17, detailed comments on my ‘The Religion Business’ book (ChatGPT).

C2. Strange assertions e.g, Q3, ‘the penguin shivering in the African sun’ (Bard)

C3. A tendency to ‘make up what you don’t know’: Q7, review of ‘Speak No Evil’ film (ChatGPT); Q15, National Apple Peeling Association (Bard); Q17, Review of ‘The Religion Business’ book (ChatGPT).

C4. A failure to find data on a subject easily found via a search engine: Q15, National Apple Peeling Association (ChatGPT); Q17. ‘The Religion Business’ book (Bard).

C5. Limited ability to display graphical images: Q13, Pythagoras’ Theorem (ChatGPT and, to a lesser extent, Bard).

C6. You get what you ask for. Several times – review of ‘Speak No Evil’ (Q7); explanation of Boolean differential calculus (Q12); spelling of parthenogenesis (Q20) – the answers were not as detailed as I had hoped but, retrospectively, this was because I did not ask the right question.

C7. A lack of source references. Bard’s answer to my National Apple Peeling Association, Q15, was the only response that referenced the source of information.

I am sure both OpenAI and Google are very aware of these limitations and are taking steps to address them.

So, what to make of all this? There is huge speculation in the media about how generative-AI chatbots will change our lives and I’ll let you dig down and find the pundits. My own opinion is that we are entering a phase of human societal development where fundamental changes are about to occur and we should both embrace and control it. Every time there is a major shift in something fundamental to our existence, there are doomsayers and enthusiasts. In the 19^th century, the Luddites destroyed mechanised looms and knitting frames claiming that such machines would ruin their livelihoods. Mechanisation did no such thing. Ex-weavers and ex-knitters became mechanics servicing the new machines, or joined the companies that manufactured the machines, learning new skills along the way – engineering, manufacturing, management.

Similarly, in the early 20^th century between 1919 and 1924, the Horse Association of America vehemently opposed the introduction of the tractor to replace horses as the motive force to pull farming implements. Millions of pamphlets were produced and distributed containing anti-tractors statements such as ‘a mule is the only fool-proof tractor,’ and ‘horses reproduce; tractors depreciate.’ ‘Tractors are causing farmers to go bankrupt,’ they said, but little did they know that the October 1929 Wall Street Crash that triggered the start of America’s Great Depression was only five years away and already building.

These days, technology opposers (neo-Luddites, so called) are all around us – those who oppose genetically-modified crops, for example, or the rise of the anti-vaxxers during the Covid-19 pandemic (and back in the 19^th century when smallpox vaccinations were first introduced), or those who shun the use of smartphones/tablets/laptops – any personal processor-based gizmo – or have a desire to return to a less-complicated life and opt to live off-the grid, or oppose the development of nuclear power stations, etc.

Ever since we started to evolve, there have been the antis and, of course, they play an important role in curbing the enthusiasm of the technologists, acting as a check and balance on developments and causing us to pause and think about what we are doing to ourselves, our families, our local communities, and the wider social classes all the way up to global considerations. Climate change activists are a prime example of global concerners.

Generative AI is very much in its infancy but, make no mistake, it’s here to stay and will be further developed. All my professional career has been spent in the electronics industry. I have seen, and even been a part of, innovations within the design and test of electronic components and systems that use these components. I have worked alongside software programmers and hardware designers and witnessed the enthusiasm with which they work and the intellectual skills they possess. AI has been on the periphery of my particular skill set but AI pioneers such as Alan Turing in the ‘50s and Donald Michie in the ‘60s and ‘70s have periodically come to my attention and kept my interest alive. In the ‘70s, I was a lecturer in digital electronics in the Electronics Department at Southampton University and I recall discussions with fellow lecturers about how to train a prosthetic robotic hand to pick up a chicken egg without either breaking or dropping it – a task very easily accomplished by a human but, in those days, difficult to achieve with a motor-controlled robotic hand. We wondered if it would be better to use microcomputers to control the motors such that, in real time, the sensors would monitor the finger pressure, passing the data to the software which would return a result that ensured the pressure was always kept between that needed to grasp the egg and that which would crack the egg. Given that not all egg shells are of the same thickness and composition, we speculated that there would be a range of these two extreme pressure points and that AI would be involved in the training process. That was fifty years ago.

My personal interest in generative AI is twofold – research and education. Ever since my PhD study period, late-‘60s, early-‘70s, I have enjoyed researching a topic that caught my interest. I welcomed the development of the Internet and, particularly, search engines such as Google which provided me with a massive index of what was available via the World Wide Web. Google, not the first search engine but certainly the game changer, opened up Aladdin’s cave of data, information and knowledge for me. Nowadays, Google has competitors – Safari, Bing, Yahoo, etc – but there’s an even bigger competitor now – ChatGPT, Bard, Tongyi Qianwen, and more to come, I’m sure. First generation search engines, of which Google is top dog, answer queries by listing relevant websites that may contain the answer to your question. It’s then up to you to enter individual websites and determine whether it contains what you are looking for. ChatGPT and Bard do this for you. You may have noticed that in my penultimate question, I asked Bard if DialogFlow was a forerunner of Bard. The answer came back – yes – along with a more detailed explanation of why. Prior to doing this, I had asked Google the same question. Google returned 9,670 results, some of which looked relevant, others not so:

Bard’s response was immediate and easier to digest:

It doesn’t need a crystal ball to predict that very soon Bard will be integrated into Google providing a powerful new way of responding to search enquiries. ChatGPT is already a part of Microsoft’s Bing search engine. Of concern however, will be the unwanted abilities of chatbots to present incorrect information (misinformation), or fictional (invented) information, or denial of any information despite it being readily available on at least one bona fide website – all of which came up in my investigations above. Two fundamental questions are: how will we know we are looking at such responses and how easily can the results be manipulated for nefarious fake news purposes such as election results, war propaganda, research findings, and anything else that might change the way we understand and react to topics that concern us?

The impact on education also interests me. A few years ago, we saw the rise of essay-writing services providing opportunities for high-school and university students to cheat. Counter-measures, such as plagiarism checkers, have been developed to identify essays that simply copy and paste from online encyclopaedias and other such sources but concern is now being expressed about output generated by chatbots that are not direct copy-and-paste lifts from source websites but which could be accused of simple word massaging; what we might term as pseudo-plagiarism. My experience (Q16’s discussion of one of my books based on the book’s blurb) is that Bard, in this case, took care not invite such criticism. In other words, generative-AI essays could turn out be very difficult to identify by plagiarism checkers.

(If this is a concern, my tongue-in-cheek solution is to introduce a few grammatical errors into the copied-and-pasted generative-AI text! Both ChatGPT and Bard produce well-structured sentences with no spelling mistakes or other glaringly-obvious syntactic or semantic errors – none that I’ve noticed, anyway. Many humans don’t! This will only be a short term solution however. If you consistently change the correct spelling of parthenogenesis to pathogenesis, there is a chance that, eventually, the chatbot will accept the incorrect spelling as a correct alternative form of the word.)

What about examinations, supervised or online? Should all supervised examinees leave their smartwatches, smartphones and tablets at the entrance to the exam room? If an examinee is taking a timed test online, should he or she confirm, hand on heart, they are not using a chatbot to determine answers to questions? How can the use of chatbots be policed if their output appears to have been generated by a human?

I contend that these are the wrong questions to ask. Examinations based on, ‘Can you regurgitate that which you have learned in the classroom or lecture theatre?’ are an artificial way of determining academic achievement. It goes back to what we do as adults when we want to find out about something, either just out of interest or to solve a specific problem. We look it up in books or, these days, use a search engine and scour the Web. Google gives us a portal to things we have forgotten or things we want to learn about. The skill we provide is knowing what is true or false and making use of what we discover. I have probably forgotten 90% of what I learnt in the classroom at school and in lecture theatres at university but with the help of Google, I can recall the detail and apply it to the problem at hand. This should be the purpose of examinations at school and university level. ‘Can you discover what you need to know and then apply it to your problem? If you can, you pass. If you can’t, dig down until you reach a level you understand, and then build on it until you reach the level where you can apply what you’ve discovered and thus solve the problem. Use all the tools available to you.’ This is what we do in real life. Educational courses and associated examinations should mimic exactly how we learn and problem solve in real life and schoolroom classes and university lecturers should be focussed on the skills necessary to do this.

If you accept this change of thinking, the fundamental concepts of education also change. No longer do we learn the times-two table by rote. No longer do we enter an examination room and sit in silence for three hours while we struggle with an examination paper that asks us to answer any four out of a selection of seven questions, or write a critique of a book we’ve been instructed to read as coursework, or translate a paragraph written in English into another language, … Ever since we picked up a stone and realised we could use it as a hammer to build or a weapon to hunt, we have built and used an ever-increasing sophistication of tools to solve even more sophisticated problems. ChatGPT and Bard are the forerunners of a brand-new set of tools that will allow us to rise above mundane tasks and invent even more sophisticated tools. The challenge, as Musk, Wozniak and their co-signatories have said, is to ensure that we do so for the betterment and not the detriment of the human race.

(^_^)