How do you grade the future?

.
In my project course "Future of Media" around 50-60 students work with a different topics every year. This year's topic is "The Future of News/News of the Future". These students are divided into 10 project groups or each group work on developing a scenario for the future.

As we are nearing the end of the term and the end of the course, each group has to (as part of the examination):
- Produce a text that will become a chapter in the jointly produced book.
- Develop a design representation of the results. This could be done in the form of a mock-up, a prototype, a short film, a website, or through visualizations of visions and scenarios.
- Develop an oral presentation that is part of the joint public presentation at the end of the term.

That is all and well, but, how do we evaluate and grade the work that these project groups do? In the course PM, it is written:

"Since this is a project course and the nature of the task and the results are very open-ended, it can be difficult to examine (judge, grade) the final results. It is not possible to specify in detail and in advance exactly what conclusions project groups should arrive at without at the same time limiting the ability to freely explore those aspects of the theme that you find to be most interesting"

Having said those nice words, how do we in practice evaluate and grade the work that these project groups do? This is a question that we have struggled with more or less since the course's inception ten years ago. We have of course had criteria, but after having used a set of criteria for two years, I was quite dissatisfied with them after last year's run of the course and I wrote a blog post about it back then.

We have now revamped the criteria (sometimes just slicing the criteria we already use in new ways and sometimes adding or subtracting stuff that was hard to judge or deemed less relevant (or even irrelevant). Below is our new-and-improved list of criteria that we will utilize at the end of the course a couple of weeks from now. The list has been published in the course blog and this means that students have the chance to evaluate and use these criteria to improve the deliverables they are working on, i.e. a win-win situation.

Some of the criteria below are coupled to a specific deliverable while others are more general. A few criteria even relate to the whole package and possible synergy effect between all the three deliverables.

----

Criteria 1 - Process. Running work that you have done since you were divided into groups and starting with the project plan and finishing with your last weekly status report on Friday next week.

Criteria 2 - High quality text. The text (book chapter) should be correct and easy to read (worst-case scenario: a text that requires a lot of effort to be understood). The text should furthermore have a well-developed line of reasoning and analyze, reflect and argue for whatever it is you want to say (and it's much better to say a few things clearly than to raise too many different issues that point in different directions). The text should be coherent and with no internal contradictions. To explain and exemplify is fine. To identify, categorize, differentiate, contrast, combine, modify, conclude (etc.) is better.

Criteria 3 - Creativity. Your project (your Big Idea) will hopefully have a lot of "innovative potential" ("idéhöjd"). To what extent are the results of your work innovative, original and perhaps surprising? Are you onto something interesting and have worked in a creative way to "solve" the problem/challenge of your choice? Does your solution meet real needs? Does the underlying idea raise the pulse?

Criteria 4 - Grounding. To what extent are the project group's results credible? Are your solutions backed up and strengthened by literature you relate to, empirical material you have collected or own experiences that are relevant?

Criteria 5 - Professional design representation. Your design representation (most often a film but other forms are also possible) should be characterized by a high level of professionalism and craftsmanship. Does you design representation communicate the concept (your Big Idea) well?

Criteria 6 - Professional presentation. Your presentation should be characterized by a high level of professionalism; you have to be able to clearly communicate your message (your Big Idea) to the audience. Was the presentation well structured, was it fun and did the presenter(s) do a good job? You should also be able to provide good answers to potential questions you get from the jury.

Criteria 7 - Credibility. How easy is it to understand your solution? Are your conclusions/solution believable and convincing? NOTE: your conclusions/solution doesn't have to be probable or even desirable, but they have to be believable!

Criteria 8 - Coherence. Does the text, the design representation and the presentation cohere and interlink? Do they support each other (or do they instead pull in different directions)? Can the results be regarded as a well-integrated whole that is more than the sum of the constituent parts?

Criteria 9 (updated 2014) - Relevance (in relation to the theme). How does your scenario relate to this year's theme? How is it relevant to (future) media and media technology?

The future is by definition unknown, so how do you evaluate and grade the future? Do you think these criteria make sense and are useful or is something missing or wrong? Do comment if you have any opinions (or praise?). Any feedback is welcome since this really is a tricky thing!

.