OFFICE WATCH

Our 30th year of watching Word, Excel, Outlook and PowerPoint.

20 August 2025 – Vol. 30 No.32

Read the full article • Share this on Facebook – X/Twitter – Threads – BlueSky

Just like Copilot or its parent ChatGPT, the Copilot function in Excel use for analysis but not so much for getting hard facts. Our testing of the new Copilot() feature shows that no-one should trust what AI says is true.

We’ve taken Microsoft’s example and extended them a little to show the real-world pitfalls and tricks for using Copilot() in Excel. It wasn’t hard to find factual errors in Copilot() responses, some big, small or not understandable!

Some lessons we learned from Copilot()

• Copilot has a slippery and changing concept of ‘truth’.

• Carefully word the prompt and context.

• Carefully check results.

• Sorting has to be done as part of the Copilot prompt, but isn’t always correct.

• Filtering to exclude some results, individual or as a group, can be done in the prompts

• Copilot has trouble parsing first and last names with a middle initial.

• Headings for Copilot() lists may or may not appear. Better to be specific.

Airports

Microsoft’s Copilot() example shows how to get a list of airports.

Source: Microsoft

Like most Microsoft carefully chosen examples, if you do a little digging the problems arise.

We add a filter by population and asked for more details “Airports in cities over half million people, show airport name and code”

As you can see, Copilot() returns a dynamic (spill) array which can include multiple columns.

However, there are problems:

• Gold Coast/Tweed Heads has a population of over 700k and it’s airport should be on the list.

• Canberra and Newcastle have populations just over 500k and should have been included.

• The proper name is “Sydney Kingsford Smith Airport”. Changing the prompt to ask for “full airport name” gives a more accurate result.

  • Just one example of how careful wording of AI prompts is important.

Which only confirms what we’ve said about AI for some time:

Always check the facts and be careful about the wording of prompts.

Another factual error

Just another factual error we found in our testing. Asking for ” Airports in cities over half million people, show airport name and code” for the UK might seem correct but it’s not.

London has 5 or 6 airports (it depends). However, you define “London airports”, it should at least include Gatwick (LGW) and London City (LCY). Luton, Stansted and especially Southend are also called “London airports” with a certain generosity of spirit .

Copilot makes the same mistake with New York, only listing JFK and not La Guardia (LGA).

But change the prompt to ask for distance from a location and suddenly Gatwick airport appears! LCY, which is even closer to Greenwich, is still missing.

This isn’t pedantic nit-picking, it’s examples of a common problem with the current AI systems. We rarely get a ‘factual’ result from Copilot or ChatGPT that doesn’t need some changes.