The challenge with a classifier constantly analysing a user’s environment is how to make it helpful without overloading the user. With all the external stimulus that humans experience in a day, if this classifier were running on a phone, the user would receive constant notifications. If this were on eyewear, the user would be bombarded with possible tens of different pieces of information at any given time.
For both cases, a couple of things could be done to minimize the amount of information shown. First, the classifier could ignore commonplace items like cars, houses, silverware, etc. Since the user most likely already knows what these items are, it would be redundant to tell the user. Second, the output of the classifier could be run against an interest classifier to see if the user would be interested in it. Today, with targeted ads, finding out an interest list for a person and matching that list with potential things that they might find interesting is relatively straightforward. While this would reduce the clutter, there would be a chance that some objects the user might be interested in would be missed.
Third, the user could be in control of what gets shown to them. This could be done a few ways. A reticule could be placed in the center of the screen/glasses that would only show information object the object it is covering. This means that the user would have to actively look at something to get information about it. The downside would be any passing objects that they didn’t get a chance to see would be lost. Another way would be for the user to physically point at objects they want information about. This is a more natural gesture, but more cumbersome for adept users and has the same downside as the above option.
Fourth, people (excluding celebrities, politicians, and those in the public spotlight) would not be classified. This would both reduce clutter and avoid privacy concerns. It would also make conversations slightly awkward if there was no introduction and two people just knew each other’s names.
Aside from general clutter, there should be some thought as to how the information is displayed. Since the user is looking at a lot of things, the info will change rapidly. If not done correctly, this could result in eye strain, headaches, nausea, and even seizures. The number of times information changes on the screen (ie, looking at a new object) should be limited to about 5 times per second. Any less than that, and the user won’t even be able to read the information.
Finally, advertising should be restricted to basic, textual overlays. For example, when a user looks at an object they want to buy, a YouTube review should not start immediately playing in the user’s ear. It would be like a hidden chrome tab suddenly playing audio. Additionally, ads should be limited to a certain number per hour to make sure the user does not get annoyed by the phone/glasses app and stop using it altogether.