4. Evaluations

Evaluations are performed using the MediaWorks DX web client, as described in this section.

4.1. A Word about Permissions

Evaluations can only be performed by users who are logged into MediaWorks DX with a NexLog DX-Series User account that is a member of the Admin, Group Evaluators or SuperEvaluator permission groups. Group Evaluators can only evaluate calls for agents in the agent groups for which they have been designated as a Leader. Group Evaluators who are group leaders can only view completed evaluations for their agent groups, and Group Evaluators who are not a group leader can only see completed evaluations about themselves.

So, as an example, if User JDarbon is:

  • associated with Agent id agent001

  • AND is Leader of Group EMT Dispatch

  • AND is in the permissions group Group Evaluators

Then, they can:

  • Evaluate calls made by any Agent in the EMT Dispatch group

  • See completed evaluations for any Agent in the EMT Dispatch group

  • Add an Agent Comment to evaluations of calls they are taken as agent001.

Agent Group Edit Page

Fig. 4.1 Agent Group Edit page

::Note: Admins and SuperEvaluators can evaluate anyone, can see all completed evaluations, and can run reports for all agent groups.

Agents don’t need an associated User account, but if your forms use Single Comment Mode or Multiple Comments and Actions mode, they will need a User account that is part of the Agent permission group; this will allow them to view the Completed Evaluations tab in MediaWorks DX software to see only their evaluations and allow them to add a comment, acknowledge other comments and digitally sign their evaluations. Note that in Single Comment Mode, no one but the agent who was evaluated can add a comment to completed evaluations.

4.2. MediaWorks DX web client

MediaWorks DX web client can be navigated to as follows:

http://(Recorder IP Address)/client/mediaworks

Log-on to MediaWorks DX using your NexLog user account ID and password.

Select a call from the “Search” or “Browser” tab, right click on the call, and then select the Evaluation form that you wish to run (only published forms will be shown in this list). The Evaluation form will open in its own tab.

Evaluate Records Sub-menu

Fig. 4.2 Evaluate Records Sub-menu

Sometimes a single phone call or interaction can end up as two records on the recorder. For example, an analog line set to record based on Vox Threshold may have a call broken in two because of an extended silence in the middle. In order to properly evaluate the whole call, you can select multiple records from the search or browser tabs, by ctrl+clicking additional pieces of the call, then right-click and select the form, as follows:

Evaluate Multiple Records as One

Fig. 4.3 Evaluate Multiple Records as One

4.2.1. Evaluations Form

Once selected, the Evaluation form will open in its own tab:

Evaluation Report Form

Fig. 4.4 Evaluation Report form

Using the form is largely self-explanatory.

Use the transport controls at the bottom of the page to listen to the call(s) while you are evaluating.

Edit the title

  • The default title will include the Agent_ID the call is tagged with and the Media Start Time. You can leave this as is, or change it as much as you like.

Select the Agent Group

  • Click the Group menu to select an agent group, and then the Agent menu will only show Agents who are in that group.

Select the Agent

  • If the call selected was automatically tagged with an Agent via the NexLog workstations feature, then the Agent and Group will be pre-populated with the Agent that was tagged, but you can override this manually. Otherwise the agent will be preset as ‘[Not Set]’ and the evaluator will need to select the agent using the dropdown menu. Do this by listening to the call and determining the agent based on the agent’s voice (or use external information such as work schedules to determine which agent was at the recorded position).

Timeline Toggle Button (highlighted in red)

Fig. 4.5 Timeline Toggle Button (highlighted in red)

Play the Call - The call(s) being evaluated are playable from the playback controls at the bottom. It will playback the call(s) in mixed mode by default, but if you want to see the calls in the standard timeline/callgrid view, you can click “Toggle Evaluation View” to see the view familiar from the Search or Browse tabs.

Annotate the call (Optional) - You can use the text and audio annotation features of MediaWorks DX to make Evaluation specific annotations that will only appear in the Evaluation and not in other contexts in MediaWorks DX. This may be used to show exactly where an Agent didn’t follow protocol correctly, or to highlight a particularly positive interaction worth noting.

Evaluate the call - Use the mouse to select the answer for each question.

Enter as much text as you like into the comment boxes to explain the score or to comment about the call. If a comment is required, you will not be able to submit the evaluation until it is entered, but you can save the evaluation as “in progress”.

If a question is not applicable to the specific call, and if the form makes an N/A available for the question, then you can click the N/A radio button to have the question ignored for this call. (N/A buttons are optional and are configured by the form creator on a per-question basis).

4.2.2. Evaluations and Saving

Once you have completed an evaluation, you can save it as complete by clicking Submit Evaluation. If the evaluation is incomplete, an error message will pop up explaining why the evaluation cannot be saved. For example, if you overlooked a question, it will prompt you to answer that question.

Evaluations can also be saved “in progress”, with the Save As In Progress button. Clicking this will bring you to a page that says “Form saved but incomplete”. You can resume all in-progress evaluations by loading the Evaluations tab in MediaWorks DX; if your NexLog DX-Series user account has any in-progress evaluations, you will see them listed on this tab with a status of In-Progress. Once an evaluation form is submitted, you will be shown the completed evaluation page for that completed evaluation.

4.3. Completed Evaluations

Completed evaluations can be reviewed in both MediaWorks DX and the NexLog Configuration Manager. MediaWorks DX has a new kind of tab, Completed Evaluations, which can be opened from the File->New Tab menu. In the NexLog Configuration Manager, Completed Evaluations are found in the Quality Factor Software section.

The Completed Evaluations page differs in MediaWorks DX than in Configuration Manager in one important way: it has a transport control at the bottom of the page that allows you to listen to the evaluated call(s).

Completed Evaluations can only be seen by users logged into NexLog Configuration Manager or MediaWorks DX with an account that is a member of the Agents, Admin, Group Evaluators or SuperEvaluators permission groups. Admins and SuperEvaluators can see completed evaluations for all agents, but Group Evaluators can see only those involving agents and agent groups for which they are a designated Group Leader, or which are of the agent they are associated with. Agents can only see Completed Evaluations tagged with their account.

The Completed Evaluations page lists all of the completed evaluations on the logger, with a heading for Agent, Form, Evaluator, Date and Score. To view an evaluation, simply click on it to select it and then click View Evaluation and the evaluation will be displayed. In MediaWorks DX, it will open in a new tab. If you want to print an evaluation, select it from the list and click print evaluation; the selected evaluation will open in a new tab in a printer friendly format, which you can then print with your browser as you can any web page.

Completed Evaluations Filters

Fig. 4.6 Completed Evaluations Filters

You can filter the list by evaluation date, media start date, agent, group, evaluator, and any by combination of these. For example, you may want to see all evaluations completed between September 1, 2011 and October 31, 2011, of agent agent003, by evaluator JDarbon. Or you might want to see the evaluations that were made at a different time of calls that were recorded September 1, 2011 and October 31, 2011. In the first case you set the Evaluation Start Date and End Date, in the second, you would set the Begin Media Start Date and the End Media Start Date.

The top of a completed evaluation form will look like this:

Top of Completed Evaluations Form

Fig. 4.7 Top of Completed Evaluations Form

It contains a summary of the evaluation, including the call records evaluated, as well as the final score and pass/fail information. Below that, it will show a breakdown of the score based on skill group. It also lists any flagged questions and any autofailed questions, and below that it shows the full completed results.

Score Breakdown of Completed Evaluations Form

Fig. 4.8 Score Breakdown of Completed Evaluations Form

4.3.1. Multiple Comments With Actions Mode

The Completed Evaluations page also has a field for Comment, which in Single Comment Mode can only be filled out by the Agent that the Evaluation is about. The Agent in question needs to be configured such that it has an associated User account that has Agent, Group Evaluators, SuperEvaluator or Admin permissions.

At the bottom, below the questions, is a section for comments and actions. This is where an Agent can comment on their evaluations, and other actions can be performed by users with the correct permissions, such as Lock, Protect, Reopen.

Evaluation Actions

Fig. 4.9 Evaluation Actions

Evaluations can exist in five states:

  • In Progress: The Evaluation has begun but was saved in progress. It can be completed by the Evaluator who started it and it does not count towards any reporting.

  • Completed: The Evaluation is complete and can be seen by the Agent involved, their Group Leader and by all Super Evaluators and Admins.

  • Protected: The Evaluation is protected and can no longer be seen by the Agent.

  • Escalated: The Evaluation was protected at completion time to flag it for the attention of a Super Evaluator or Admin. Like Protected, it can no longer be seen by the Agent. A Super Evaluator or Admin can decide to change the status to locked, protected or unprotected.

  • Locked: The Evaluation is locked from further changes being made. It can be unlocked by a Super Evaluator or an Admin.

4.3.2. Single Comment Mode: Agent Comment

The Completed Evaluations page also has a field for Comment, which in Single Comment Mode can only be filled out by the Agent that the Evaluation is about. The Agent in question needs to be configured such that it has an associated User account that has Agent, Group Evaluators, SuperEvaluator or Admin permissions.

Completed Evaluations Form with Agent Comment

Fig. 4.10 Completed Evaluations Form with Agent comment

4.3.3. Deleting Completed Evaluations

Only Users with Admin permissions can delete completed evaluations. On the completed evaluations page, Admin users will have an additional button Delete Evaluation. If an evaluation is selected from the list, clicking this button will prompt to make sure you want to delete the evaluation. Clicking ok will then delete the evaluation.

Deletion Button for Admin Users

Fig. 4.11 Deletion Button for Admin users

Deletion Confirmation

Fig. 4.12 Deletion Confirmation