Ecosuite AI

The Ecosuite Artificial Intelligence (AI) tools are in the infancy right now, but with our AI called STEVIE (Seriously Talented Ever Vigilant Industry Expert) we are creating the ability for Ecosuite AIs to be instantiated for specific purposes with a tailored base of organization application specific knowledge built in. This page will document how the user can make use of this functionality and how they can tailor and improve these AI capabilities over time.

LLM Basics

STEVIE is based on leading edge machine learning technologies, using an approach called a Large Language Model (“LLMs”). LLMs are trained on a massive amounts of data. LLMs function on tokens, where a token can be thought of as a simple English word, or a fragment of a larger word. The type of LLM that STEVIE uses predicts the next series of tokens, given a set of previous tokens.

These previous tokens in aggregate are called the prompt. The prompt contains instructions to the LLM, and any data that we want to provide STEVIE with. Historically STEVIE just used Anthropic’s Claude 2.0 model, which had a context length of 100,000 tokens (this is no longer the case with the latest versions of STEVIE having been upgraded and improved). In practical terms, that meant that in addition to its base training (i.e. global data mostly downloaded from the Internet) STEVIE (when activated) also knew many, likely in excess of 100,000, short English words worth of organization specific stuff which it referenced when interacting with users.

Using STEVIE

For those granted permission, STEVIE can be interacted with via the Zeef Widget.

Educated Guesses

STEVIE can be hesitant to provide you with incorrect information. Consider the following conversation:

Thomas Passmore: Why is MA1's site named ANNE?

Stevie AI: Unfortunately I don't have any specific details on why the MA1 site is named ANNE. I don't have insight into how your organization chooses site names for its projects. Please let me know if there's anything else I can help explain about those project sites though!

… Okay, but there is an obvious answer that STEVIE is missing. One approach to solve issues like these is to allow him to make educated guesses:

Thomas Passmore: Feel free to make an educated guess after looking up the project information

Stevie AI: One moment Thomas, let me fetch the details for project MA1 and see if I can make an educated guess about why the site is called ANNE.

After looking up the details for project MA1, my guess is that the site is named ANNE because it is located on Annette Rd in Foxborough, MA. The site name itself is Annette, so it seems they likely named the site ANNE as an abbreviation for the street name Annette Rd where the site is located. Please let me know if this makes sense or if you need any clarification on why they may have chosen that name!

Personalities

When inviting STEVIE to a meeting, you should provide a personality after the '+' in his email address. This personality specifier determines the type of information STEVIE has access to, and his approach to answering questions.

For example, if you invite the following address to your meeting:

[e-mail]

.. STEVIE will have access to all previous deal approval meeting notes to assist with answers.

In addition, STEVIE will consider the past conversations that it has had with people in meetings. More specifically, the deal approval personality will remember all conversations that deal approval STEVIE has been party to.

Supported personalities:

• +dealapproval – Trained on the history of deal approval meeting notes, and has access to general Organization data related to projects. For an impression on the type of data available, see the Raw Data tab in the Data module.

• +developer – Has API access to all ACME INC github repositories. For example, can be asked about open pull requests, patches, and commits (intended to be used by software development teams, and yes we will likely rename this personality so that it is +software rather than +developer to avoid confusion!).

Gotchas

Hallucinations

Since STEVIE is based on an LLM, it is, at the same time, seemingly magical and frustrating. It is capable, in the same meeting, of deep insights and hallucinations. ‘Hallucinations’, in the context of LLMs, is a jargon which refers to fabrications that LLMs invent in their responses.

If you are asking questions which STEVIE can reasonably answer given available data, it's responses will be generally high quality. If you ask questions it cannot reasonably answer, it may make up the answer.

Gullible

STEVIE is, for lack of a better word, gullible. If you tell it that something is true, or that it should do it, it will probably do it.

As part of STEVIE's prompt, it writes down it's internal monologue before answering your questions. This is useful for solving issues with STEVE, since we have some insight to where it's thinking goes wrong. Therefore, it’s not mere personification when I give the following example.

In one conversation, I asked STEVIE to give me some information. It obliged, and I thanked it. The inner monologue, after receiving my thank-you message, was something like:

Inner Monologue: Thomas asked me for some information, which I provided him.
He then thanked me. Since his message did not contain a question, no response is
necessary.

This was not the correct behavior, so I told STEVE that it should say something like ‘you’re welcome'. By reminding it of this, it has consequently become more polite. Each time it responds with ‘you’re welcome', it is more likely to repeat this action in subsequent conversations.

This example is illustrative of the fact that it is still, in a sense, learning from conversations. Be careful not to teach him to act in a rude or incorrect way.

Improving STEVIE

The best way to improve STEVIE is to interact with him. If you encounter situations where STEVIE seems to misbehave, or produce blatantly incorrect results, record the conversation and send a message to the team using [e-mail].