I think the study needs some more prompting the people flow has slowed down some. I will see if I can think up new places to advertise and hit up my family when I see them.
Speaking of family I will be out of the office around lunch tomorrow to visit some with them.
I was thinking of something like the above as the main page. The big features are an agent selector on the upper left to allow the user to the design multiple agents at once. The top panel will have agent specific information on it, right now I have it displaying which set of subsumption is displayed and allowing the user to switch between them however they should probably include help file access and a button to bring up and alter the agents initial properties. The agent designer buttons are store on the lower left and include save, load, test, execute. The main area of the display will of course be filled with the subsumption arch. I envision this panel as being entirely mouse interactive. An option menu of new items to add to this screen will be displayed via right click. Mousing over present objects will present information about them. For example the diamond triggers will present a detailed description of there triggering conditions. The rectangular central behaviors will give a description of the behavior to execute. Mousing over will provide a more detailed description. Finally mousing over the third square inhibitor will show what effect this layer being active will have on the lower layers via a modified/color coded horizontal bar graph.
I would like to use a common map point interface for all behaviors that require it. This interface would show the map of the area the agents will interact with in the left hand side and then present any controls to the user on the right. This will be useful at least for setting up patrol paths and defining starting points but i expect that more behaviors will be require some form of spatial definitions.
To Be continued.
Things to Look at:
Spatial Calculus
Virtual Human Markup Language
Human Markup Language
Subsumption Arch
I have a solid idea of the unique contribution I would like to present and at IUI and look foward to doing the research on it. Primarly it should be a presentation of the agent design tool and a user study to back it up. The study would be the standard use to the tool to make an agent that does this kind of thing. I would like to devise a metric by which we can automatically rate the quality of the agents the users generate and produce a numeric quantitative evaluation. I also think a comparison using such a system against another design methodology would be in order. I would recommend testing against FSM since they are commonly used by most of the other agent design systems. It shouldn’t be that hard to produce a system that can do both subsumption and FSM. I would like to continue some of the work I read last week and evaluate how much the users “trust” the agents they produced and see what there confidence levels looked like. This concept of trust might prove important given that we would like to submit DASSIE as a training tool for potentially life threating situations. I will need to check our IRB paperwork on what questions we can ask. Finally the work presented here regarding providing the users with rich levels of feedback was very interesting and I would also like to integrate some of it. I feel like we could provide a sort of live demo of the agent that would provide a real time marking of which state is active while the agent moves through the world. If possible it would be nice give the participants the results of the quantitative analysis at this point as well. The participants could then use this to edit there agent design. It would I think be interesting to run participants through both with and without this additional feedback.
Also there are demos scheduled for 2:00 tomorrow thursday.
Implementation: 2D interface using python and the widget library to build the agent. Agents are simple and the engine behind them isn’t the full DASSIE agent core(target completetion for interface (19th August). Agents are then ran in a 3D environment(Panda) and objectively evaluated on task completion. The enviroment should be one easy to construct but similar to our DASSIE study enviroments. Possibly the COMET enviroment that Shawn has been working on would work. The pre defined task would be related to or one of the desired DASSIE tasks.
Also possible other things to look at. Trust or confidence in the agent – User rate themselves based on how well they think they did.
Similarity between generated agents.
User metrics, time spent on instruction screens, time spent building agent, ect.
AI designer will incorperate the first round of results from the ontology study.
Study must start no later than first week of November.
It is basically 2 months before the IUI deadline and I have went through most of the papers they published that are at all relevant to what we are doing. I noticed a couple of common themes running through the accepted papers. Pretty much all papers have a human study attached to them to showing that that whatever the paper is about is a good thing. One of the major differences between the 4 page papers and the 10 page ones seems to be the quality of the study that is attached to them. Papers generally fall into one of three groupings: Self adapting interfaces, Machine learning algorithms that improve interfaces, or interfaces that provide revolutionary ways to assist, or generalize a task to broader groups then normal. The last category seems to be a near perfect fit for the DASSIE stuff which is fortunate. I am starting to get a good feel for what the current state of the art in intelligent interfaces and plan to move my reading back into the AI areas next.