Think twice before enabling Bing Spell Check at LUIS level

Hi,

Enabling Bing Spell Check, one of the Azure Cognitive Services at LUIS level is a piece of cake, indeed, this involves the following steps:

  • Getting a key
  • Registering the key in LUIS
  • Associating that key to your LUIS application

Once done, you also need to change a few things in code:

[LuisModel("appid", "appkey",LuisApiVersion.V2,null,true,true)]


where you specify via the last value of the LuisModel attribute that you want to enable spell check, which allows you to retrieve the altered query by code:

result.AlteredQuery

if any correction took place. That is fairly easy and requires almost no effort. Moreover, you can easily test it in the browser:

https://westus.api.cognitive.microsoft.com/luis/v2.0/apps/yourapp?subscription-key=yourkey&spellCheck=true&q=howw to get onedrive

and play with true/false with the spellCheck parameter for the same utterance. You’ll notice a difference in intent resolution (usually the probability varies), which means that LUIS interprets the corrected utterance and forwards it to your bot. So far so good, this looks perfect but if I write this blog post, it’s because there is one major shortcoming (at the time of writing): this has an impact on entity extraction.

A concrete example of that is the following:

how to access SmartIT?

with spell check enabled, the word SmartIT will be splitted into two words: Smart IT. In my particular context, SmartIT is the name of an application and therefore, this is not captured as an entity anymore.

In my particular case, SmartIT is part of a phrase list feature so I had expected that Microsoft would somehow exclude such words from the translation process since they were explicitly specified by the LUIS application owner.  Unfortunately, at the time of writing, they do not pair the translation process with the phrase list features.

So, what are the alternatives? There is no black or white answer to this question as it depends on your particular context. If you’re using very strange names in your entities, you’d better not let LUIS interpret the translated utterance as it would result in transforming your entities into something else. Otherwise, some alternatives are possible:

  • Let LUIS call Bing Spell Check but do not at intent resolution time. For this to happen, simply turn the spellCheck switch to false in the LuisModel attribute. Doing this will still perform the translation but LUIS will interpret the original utterance and will make the translated one available via the result.AlteredQuery property.
  • Do everything by code.

Feel free to watch my videos on Azure Cognitive Services

Happy Coding!

Advertisements

About Stephane Eyskens

Office 365, Azure PaaS and SharePoint platform expert
This entry was posted in Azure Cognitive Services, NLP and tagged , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s