What does ratemyprofessors.com actually rate?

"For almost a century, attempts have been made to evaluate the teaching effectiveness of instructors in higher education. Much of this effort has centred on an attempt to create instruments that would allow students to make this assessment. These instruments have varied over both place and time, and have created one of the longest lasting debates in higher education. ... One source stated that there were close to 3000 articles published on student evaluation of teaching (SET) from 1990 to 2005 alone. Published findings on the topic are so voluminous that many researchers have been using the method of meta–analysis, in which the case is not a subject but an entire published article. Nevertheless, little agreement has been reached on key points. ... One of the difficulties of studying SET has been finding adequate and appropriate large samples, because of the confidential and anonymous nature of the inventories for both students and instructors. An alternative has become available online. Increasingly popular databases, such as ratemyprofessors.com, uloop.com, koofers.com, myEdu.com and studentsreview.com, offer large and easily accessible sources of data. Although these sources are tempting, a question remains about their validity. ... ‘Does ratemyprofessor.com really rate my professor?' "

What does ratemyprofessors.com actually rate? (2013) Assessment & Evaluation in Higher Education doi: 10.1080/02602938.2013.861384
Abstract: This research looks closely at claims that ratemyprofessors.com creates a valid measure of teaching effectiveness because student responses are consistent with a learning model. While some evidence for this contention was found in three datasets taken from the site, the majority of the evidence indicates that the instrument is biassed by a halo effect, and creates what most accurately could be called a ‘likeability’ scale.


More on this exciting field of research

A.J. Cann

Read More

The murky world of academic anayltics

I've been involved in a number of discussions this week about the use and abuse of metadata. The reactions have been mixed. Many people seem unaware of the potential dangers in this area, some of them who clearly should be better informed. Others are sanguine about the prospects. The final group have their tinfoil hats clamped securely over their ears - obvious nutjobs.

Or are they?

Before you decide, read this recent article in The Atlantic: Your Job, Their Data: The Most Important Untold Story About the Future.

One of the things I have been thinking about this week is how much metadata is captured by academic systems such as Blackboard, Echo360 and Panopto. Most people seem to be blissfully unaware of this, or simply don't care. Stephen Walker's post Lecture Capture and Data Capture did a good job of raising the key issues. We know what use commercial services such as Amazon, Facebook and Google make of this data - but what about academia? Is "Do No Evil" the watchword or do uses such as performance management - of students or staff - take precedence?

There is an easy way to deal with this problem - clear and transparent institutional policies on these issues. But more often than not, policies are lacking, either through omission or commission, resulting in loss of trust. The University of Manchester Policy on the Recording of Lectures and other Teaching and Learning is exemplary in this regard and should be followed by all institutions:

3.2 Prior to the beginning of each Semester, teaching staff will be informed that recordings will be made if their teaching and learning activities take place in a location containing automated lecture capture technology. Staff not wishing to have their sessions recorded should respond to this stating 'Opt out'.

3.6 Recordings will not be used for staff performance management purposes.

Such clear policies allow both staff and students alike to concentrate on learning rather than spending time wondering what Big Brother will do with their personal data.


Update: Helpful link via Martin Hawksey:
Prinsloo, P., & Slade, S. (2013, April). An evaluation of policy frameworks for addressing ethical considerations in learning analytics. In Proceedings of the Third International Conference on Learning Analytics and Knowledge (pp. 240-244). ACM.

A.J. Cann

Read More

#ITFocusWeek - What's wrong with "lecture capture"? This:

Today I'm taking part in a Question Time panel about lecture capture technology. I'm cast comfortably in the role of curmudgeonly old git who doesn't like it. The reality is slightly more complicated.

Everyone likes online videos. I do, that's why my YouTube channel is closing in on 1.5 million views. At least, I like good online video. But what does that mean? It certainly doesn't mean the lowest common denominator of talking head video - plonking yourself in front of a webcam and droning on because that's the easiest thing to do. And it certainly doesn't mean any video longer than five minutes. All of which means that the idea of "lecture capture" is wrong headed.

Alexa.com reports that people average 17 minutes per day on YouTube and watch 12 videos, that's 85 seconds per viewing. Here's a graph which shows YouTube's own data for "audience retention", i.e. the length of time people pay attention to online videos before stopping or skipping ahead. What do you think this data says about the concept of "lecture capture"?

Ah, you say, but people sit on their sofas watching David Attenborough with rapt attention for an hour. Well first of all, you're not David Attenborough. Second, you don't have 1% of 1% of the budget or the time that the BBC or Sky spent on making that program. But most important of all, you don't understand the difference between lean forward and sit back media. The interactivity of an integrated control bar in online videos banishes the passivity of watching linear video on a big screen such as a television and encourages use of fast forward skipping. Online, all attention spans are short.

The Atlantic recently ran a piece titled Lectures Didn't Work in 1350—and They Still Don't Work Today. It's hard to know where to start listing what's wrong with this article. First, it's not about higher education or about lectures at all, it's about primary and secondary education where lectures (rightly) don't feature as a mode of instruction as far as I'm aware. But even if you apply it to higher education, the whole piece conveniently ignores the rather obvious fact that lectures have indeed worked since 1350 - that's why students pass rather than fail. It is simple minded twaddle to simply ignore this. And in an age where technology increasingly threatens to distance us from each other, the physicality of the lecture room increases its impact and its value. Sitting in front of a recording erodes that valuable link between staff and students.

So what's my problem with lecture capture technology? The very name. The idea that a lecture can be captured and bottled is laughable and shows a fundamental misunderstanding. This technodeterministic language cements the idea of technology replacing rather than augmenting pedagogy, the wrong-headed notion that disruptive technologies kill older technologies. This is demonstrably false - TV didn't kill radio, and radio didn't kill newspapers, etc. Instead, in the real world we have the Lindy Effect - the longer a technology has been around, the longer we can expect it to survive. Lectures are here to stay, so the question is, how do we use new technologies to augment and improve lectures, not to replace them?

And that's where the use of technologies such as Echo360 and Panopto come in useful. We use them to produce short (less than 5 minute - look at the graph) high impact videos covering the threshold concepts that students struggle with. And we use those recordings to augment and improve the vital lecture experience. Give 'em the old TED razzle dazzle. More importantly, we make these technologies available to students and we challenge them to make their own videos explaining threshold concepts. Instead of sitting passively in front of recorded lectures students become active partners in transmitting knowledge. And we stop pretending that technology solves problems. Technology doesn't solve problems, people do. But technology can cause problems, such as increasingly passive learning. Let's not let that happen. Let's encourage the use of online video. Let's stop talking about lecture capture.

But don't take my word for it:

• Jones, S.E. (2007) Reflections on the lecture: outmoded medium or instrument of inspiration?. Journal of Further and Higher Education, 31(4), 397-406
• Mulryan-Kyne, C. (2010) Teaching large classes at college and university level: challenges and opportunities. Teaching in Higher Education, 15(2), 175-185
• Mark Smithers: Is lecture capture the worst educational technology?

A.J. Cann

Read More

What happens when an economist measures teaching quality?

... probably not what you're expecting.

"Although assessment exercises are nowadays a widespread practice in educational systems, we lack an effective understanding of what is the actual feedback effect on the overall efficiency (macro level) of the system and on the individual level (micro level). Given the lack of any sort of incentive to perform better in teaching activity, I formulated the hypothesis that this effect is null (at least for the Italian academic system), implying that the overall quality of teaching which is supplied does not change over time. Even worse, I claimed that the same is true at the individual level. ... This study ... shows that the evaluation of teaching quality might not have been conducive in improving it. On the one hand, it is possible that feedback from the questionnaire was not used effectively, and on the other (much more plausible) that the incentive system is not working properly. This last argument might entail dramatic consequences for the Italian academic system (and in general for European education systems), which tends to be increasingly research-oriented and neglects teaching duties. At this point one crucial question remains open: does it make any sense to continue carrying out teaching quality assessment when it does not enhance teaching performance? The tentative answer is ‘no’, at least until some incentives are introduced for providing better teaching or a more efficient teaching performance management is established."



Bianchini, Stefano. Feedback effects of teaching quality assessment: macro and micro evidence. Assessment & Evaluation in Higher Education (2013): 1-15. doi: 10.1080/02602938.2013.842957
Abstract: This study investigates the feedback effects of teaching quality assessment. Previous literature looked separately at the evolution of individual and aggregate scores to understand whether instructors and university performance depends on its past evaluation. I propose a new quantitative-based methodology, combining statistical distributions and transition probabilities matrices, to take into account the dynamics of teaching quality over time both at the macro and the micro level. Using a three-year longitudinal panel from an Italian university, it is shown that evaluation exercises do not impact future teaching performance at either the university level or the individual level.

A.J. Cann

Read More

Blogging 2013

The LSE Impact of Social Sciences blog asked me to write a guest post about the blogging session at #solo13. So I did, and they have just published this:

---

As academic blogging becomes mainstream, science communication must facilitate depth and breadth in online discourse.

The SpotOn London conference, formerly called Science Online London, has just run for the sixth time. Held at the British Library and sponsored by Digital Science, The Wellcome Trust and a number of other high tech, high profile sponsors, SpotOn London (or #solo13 as it is more widely known) is both the flagship event for current science communication and collaboration and represents the cutting edge of developments in the way science is being conducted. The three official strands at the meeting, science communication and outreach, online tools and digital publishing, and science policy, reflect the interest and enthusiasms of those involved.

I have been fortunate enough to be involved with all six Science Online London meetings. Although I was not physically present at the first Science Blogging conference at the Royal Institution in 2008 I spent a day following the event online - my first exposure to how new media were changing academic life. The feeling of excitement around the emerging tools - Friendfeed (precursor of the present day Facebook interface), Twitter, informal video livestreams (from people's phones and of varying degrees of reliability), and aggregation of discussions from blogs, Flickr, etc - had sufficient impact to shape my thinking about and approach to online communication, as well as my own academic practices.

As a grizzled veteran, #solo13 inevitably didn't have the same impact for me that some of the earlier meetings did. This is just part of the natural cycle of exposure - adoption - commonplace with emerging technologies. What was noticeable at #solo13 was the emergence of a younger demographic of PhD students and science communicators experiencing it all for the first time. I don't intend to try to cover all the content and discussion from this packed two day event in a single post, but I would like to describe some of my reactions to the "Using blogs and other online forums for communication between scientists" session held on the second day. You can watch a video recording of the whole session online:

As I commented during the session, I'm a serial blogger who couldn't stop if I wanted to. In fact, the only major change I could envisage in my blogging practice might be to obscure my identity when writing about certain topics - more of this later. I currently maintain five active blogs. My personal blog, Science of the Invisible http://scienceoftheinvisible.blogspot.com , is where I write about my educational research and professional life, but also share more personal (but not intimate) content such as recipes and commentary. MicrobiologyBytes is my oldest continuous blog, and writing about microbiology online helps me stay up to date for my teaching and my personal interest in science communication now that I am officially no longer "research active" in this field (which is a reflection of grant income rather than the end of my scholarship). I am the Internet Consulting editor for Annals of Botany, responsible for social media policy, and our blog is very much a team effort aimed at promoting the content in the journal, but also awareness of plant science to a more general audience. @leBioscience is an online shop window, the 21st century equivalent of the staff newsletter we used to distribute in print 20 years ago. My student support blog is both a content management system and an authoring tool for advice and guidance to the students I teach, and this growing personal repository of information saves me endless amounts of time by allowing me to refer frequently asked questions to this resource rather than to have to answer each one anew. In addition to this core, project blogs and grant blogs come and go with fixed lifespans as dissemination tools and archives for particular projects. You get the idea....

With this history, it was hard for me to resist a gentle smile at the panel member who introduced themselves with the line "I've been blogging for two months", but at the same time, it was encouraging that there is more of an acceptance of blogging as an important academic tool. Unlike the Science Blogging 2008 meeting, there is now a feeling of establishment rather than revolution about blogging - part of the process of scholarship, but particularly with regard to individual development rather than as an institutionally-owned process. Few academic institutions actively encourage their staff to blog and there remains a justified wariness of the possible consequences of being judged to have frittered away valuable time which could have been "better" spent on traditionally mainstream academic activities. Across a wider sector, Marketing still retain control of blogging in many organizations and we have still to fully realize the benefits that can arise from reflective cross-institutional communication and knowledge exchange which can arise from a blogging culture. At a simple level, it would be easy for institutions to encourage more staff to blog by simply providing the tools - for example, a site-wide installation of Wordpress available to all. Moving beyond that to a culture where blogging is the norm rather than is much more complex and not as easy to achieve. In a time of financial stringency, senior academic management will need to provide the lead in such matters in order to enable this sort of change.

There is also something of a pervasive feeling of weariness around blogging. There appear to be some bloggers who don't enjoy the process but do it out of a sense of obligation or not wanting to miss out. Blogging is still a peripheral rather than a core (paid) activity for most. For me, blogging is fun (although not always enjoyable, and not as much fun as teaching, with which it overlaps) - blogging is not my day job, so it's something I'm often doing in the early hours of the morning or in snatched minutes between meetings if I can get my brain in gear. If the reward is not financial nor much institutional credit for most, what keeps academic bloggers going? One common experience at the meeting was of colleagues and students who read your blog secretly, then come up to you at some point and say "I read your blog". It doesn't count as REF impact, but this peer acknowledgement is a significant personal reward. If your institution does not provide the platform, other sites such as the Huffington Post, The Guardian and New Statesman all accept guest posts, and if you have the time and talent to get your writing accepted by such sites it will certainly attract attention within your institution and from future employers.

The issue of commenting on blogs was also discussed in the session. For most blogs, comments are dead and any conversation is now based around is now retweets and reposts to social media sites rather than a threaded stream on the original site. The same is also true of online commenting on scientific papers - on the PLOS and Nature websites for example. There is plenty of discussion going on about the content, but it is distributed across many networks rather than being focussed into an easily accessible thread. In spite of a relaxed attitude to this by some, in my opinion this is a problem as it encourages superficial commenting at the expense of more in depth conversation. Sharing on Facebook is a good thing but it is not a substitue for a good discussion thread on a thorny issue. In science in particular, putting your head above the parapet and being seen to criticise the work of senior scientists is still a risky business, even when such comments are made in neutral tones and intended as a positive contribution to ongoing work.

Overall, I came away from this session feeling buoyed up by the discussion. A blog is not a book or a newspaper, but it is more than a Twitter account or a Facebook page. Blogs are still the centre of serious online academic communication but there is still a long way to go until the Republic of Blogs is established and academic blogging moves from being a spare time activity to a mainstream output. I'm delighted to have been asked to contribute this guest post to the LSE Impact blog and I intend to expand my guest blogging activity as time permits. Sleep is over rated anyway.

Read More

Why Are We Bowling Alone?

The most influential book I have read in the past 10 years is Robert D. Putnam's Bowling Alone (Simon and Schuster, 2001. ISBN 0743219031). I'm aware of some of the criticism which has been leveled at Putnam's work, but for me, as for many others, Bowling Alone provides a description of the society I see around me. The problem I've always had, and which Putnam does not fully address, is why has there been the erosion of social capital that he describes?

Last weekend I read Thinking, Fast and Slow by Daniel Kahneman (Penguin UK, 2012. ISBN 0141033576). Kahneman's book does not have the impact of Putnam's and while not undermining the overall corpus of his work, comes across as rather superficial in many ways. But one aspect that Kahneman discusses definitely struck a chord, the issue of money priming:

“Reminders of money produce some troubling effects. Participants in one experiment were shown a list of five words from which they were required to construct a four-word phrase that had a money theme (“high a salary desk paying” became “a high-paying salary”). Other primes were much more subtle, including the presence of an irrelevant money-related object in the background, such as a stack of Monopoly money on a table, or a computer with a screen saver of dollar bills floating in water. Money-primed people become more independent than they would be without the associative trigger. They persevered almost twice as long in trying to solve a very difficult problem before they asked the experimenter for help, a crisp demonstration of increased self-reliance. Money-primed people are also more selfish: they were much less willing to spend time helping another student who pretended to be confused about an experimental task. When an experimenter clumsily dropped a bunch of pencils on the floor, the participants with money (unconsciously) on their mind picked up fewer pencils. In another experiment in the series, participants were told that they would shortly have a get-acquainted conversation with another person and were asked to set up two chairs while the experimenter left to retrieve that person. Participants primed by money chose in the exto stay much farther apart than their non-primed peers (118 vs. 80 centimeters). Money-primed undergraduates also showed a greater preference for being alone.”

Reading this instantly bought Bowling Alone to my mind. I spent a happy/unhappy 24 hours reflecting on this new knowledge (depending on how you want to look at it). Sadly, the next day I read Michael Kraus's post, A Social Priming Finding with Direct Replications, which is another take on the recently discussed "Replication Problem" in psychology.

"Social priming researchers are faced with the public perception that our field lacks methodological rigor. In the past, the dominant reaction has been to simply downplay these concerns and assure people that our findings replicate."

Together with the rather glib tone of Thinking, Fast and Slow, this has somewhat undermined my confidence that I have finally found a reason we are bowling alone. But I can't shake the gut feeling that affluence must be at least part of the explanation for the accelerating enclosure of public goods and the continued decline of social capital.

• Is it time to up the statistical standard for scientific results?
• Social capital isn't an unadulterated good

 

A.J. Cann

Read More

Time is of the essence

I don't think there was a single session at #solo13 where time was not mentioned as an obstacle to getting more of what we want.


ScienceGist (tagline: "Simplifying science") aims to demystify scientific papers by publishing easily understandable summaries. The project is the outcome of a hackday earlier this year. It's very early days for ScienceGist, and there's a limited range of content there at present, for example, Plant Biology such as this.

I like the idea of ScienceGist a lot. But there's a problem. The site is in competition with Wikipedia, reddit, Buzzfeed, etc, and compared to those sites it is a minnow. Brain Kelly quoted this yesterday:

"as Cameron Neylon had said in the “Wikimedia UK Annual Review 2012-13″:
“If you’re serious about ensuring public engagement in your research then you need to make damn sure your work can be incorporated into Wikipedia. Wikipedia is the most important engagement channel for your research."
(The Wikipedia Editing Workshop Session at the SpotOn 2013 Conference)

So far I have only reposted content I have written for other sites. Since both are published under a CC license this isn't a problem, but if I wasn't already writing this content for another site, would I have time for ScienceGist, good as it is?

One thing I've learned about curation is to find a model that works for me and not to try to do everything. In cutting back to a single channel, popularity has to be a key feature. I really like ScienceGist, but they've chosen a hard furrow to plough.

A.J. Cann

Read More

Badge Sex at #solo13

Yes, that's got a "ge" on the end of it. And if your name is Badge, this post isn't what you think (I hope).

At #solo13, Blendology (rather than Bellendology which I originally typed) supplied interactive badges that you could tap to show who you had interacted with and which sessions you attended. In addition to uncertainty surrounding the protocol of staring at ladies chests while trying to read their name badges, since the lanyards were a little long, these badges introduced the additional element of trying to introduce a device into the groin area of someone you've never met before. Certainly an ice breaker. Stephen Curry came up with the idea of "badge sex", reminding me of the epidemiology practicals we run for students. (Before too many people apply to do degrees at this university, it should be noted that no actual sex is involved in these practicals, although cherries are.) The outcome of all this tapping is a digital record of of interactions - which someone else memorably referred to as Science Grindr. Here's mine:

Although better than cardboard business cards, I'm not sure my individual data is of much use - in the sense that it doesn't add a lot to knowing someone's Twitter handle. Some of the names I don't recognize are Blendology staff "pre-taps". Hmm, I understand why they want to do that but it doesn't do much for data integrity. The overall collected data is probably much more interesting - were strands distinctive or isolated, which sessions were the most/least popular, etc.

Inevitably there were concerns raised about privacy. As far as I'm concerned, I made no secret of the fact that I was at this event by blogging and tweeting about it, so I don't have a problem with the data being published. The badges were opt-in at this event, and since privacy has a lot to do with choices, I have no great concerns.

So is this the future of conferences? Nah, let's just RFID microchip everyone to see who eats the most cake. 

#solo13 Mash-up from Graham Steel

A.J. Cann

Read More

#solo13 versus #solo14

I'm back from the SpotOn/Science Online conference where I facilitated a workshop on dark social:

Slides | Storify

This was my fifth Science Online conference in a row, and while it didn't have the impact for me of some of the previous events, it was still well worth while. Noticeably, the next generation have arrived, which is good. At the same time, SpotOn/Science Online feels like it is in need of a refresh and it is time to try something new to capture the impact of the earlier events.

Where would I like to see SpotOn go next? Ironically, I'd like to see it go all the way back to its 2008 roots and become much more focused online. The online component was the revolutionary element of the first event, but it is now standard for all major conferences. I would like to see the SpotOn conference as a one day midpoint/highlight of a month long online event.

It would be a huge mistake to try to capture all the activity in one site. Rather the focus should be on encouraging contributions across a distributed network of media - blogs, Twitter, Google+ hangouts, YouTube, ScienceGist, Wikipedia, etc, etc - conference as aggregator. Two weeks of online activity, with themes, campaigns and targets, one day f2f event (a sort of AGM), then another two week follow up to capitalize on the momentum rather than dissipating it all the Monday morning after the meeting. Build an online community of mutual support.

What say you?

A.J. Cann

Read More

SpotOn London 2013: The Dark Art of Dark Social: Email, the antisocial medium which will not die #solo13

I'm at SpotOn London 2013 today, doing this:

The Dark Art of Dark Social: Email, the antisocial medium which will not die
Email newsletters go back to the dawn of the Internet but have been neglected in recent years with the rise of shiny social networks. But eTOCs have seen off RSS and email will not die. Why else does Twitter encourage users to share tweets via email? The evidence is clear that message dissemination via social networks only reaches a relatively small proportion of the potential audience and that most link sharing occurs via private channels such as email (hence “dark social”). The Silk Road bust and the unmasking of Dread Pirate Roberts reveals how difficult it is to maintain anonymity online. Public discussion of science by scientists, e.g. via online comments, is limited by the high risk nature of the putting your head above the parapet in the hierarchical merit structure of science. Yet discussion of published science via private channels continues unabated. The rebirth email on mobile devices and the contribution of social media to information overload is causing a rethink about the utility of public versus private channels and we are seeing the rebirth of email newsletters and mailing lists. This workshop looks at the case for and against dark social, practical aspects such and tools such as how to use blogs as content management systems for email generation and asks you to embrace the dark side.

The Dark Art of Dark Social: Email, the antisocial medium which will not die
Alan Cann

A.J. Cann

Read More

Mid Mods

click for the big picture

A.J. Cann

Read More

The Great Email Opportunity #solo13 #solo13dark

At #solo13 I'm running a workshop on email: The Dark Art of Dark Social: Email, the antisocial medium which will not die. Yesterday I gave a short presentation about using email with students, but I've been thinking about email for the past year as part of my current interest in Dark Social. As a result, I was interested in a new post on AllThingsD, The Big Email Opportunity.

I don't particularly want to be in the position of being the public apologist for email, but I don't agree with Nick Morris that email is evil - although you could make a case that some of the people who use it are. More charitably, I think most of them are misguided rather than evil, and if high powered academics don't know the difference between Reply and Reply To All, who am I to correct them? What I particularly like about Robert Abbott's post is that it shifts the focus away from users onto the software. He is absolutely right to point out that email systems have not moved on since the 1990's, with the worst offended of all being Microsoft Outlook. (Man, it feels so retro bashing Microsoft).

A little dab of A.I. and almost all of what people feel to be the "problems" of email go away. Short of that, decent software that reminds the user to include a subject line (helpfully suggesting a suitable offering from the context of the thread) and automatically attaches the correct attachment for them would be an improvement. (Yeah, remember the 90s? That's how it was kids.)  Email has to change, and the sooner the better. Long Live Email! It ain't going away, so let's get it right.

A.J. Cann

Read More

Students and Email

I'm just about to give this talk as part of our monthly PedR meeting. I'll be giving a more research-focused version of this at Solo13 on Saturday. 

Video:

A.J. Cann

Read More

Getting Evo-Devo: Concepts and Challenges for Students

Useful article for anyone involved in teaching these fundamental concepts.

"Evo-devo content presents students with new conceptual challenges and potential difficulties in attempting to understand evolution. For example, while several evo-devo concepts rely on the supporting concept (SC) of conserved gene networks that operate in a variety of developmental contexts, many students hold that each trait of an observed phenotype is the result of the expression of a single gene. Improving strategies for teaching evo-devo will benefit from an inventory of concepts appropriate for undergraduates, a learning progression toward their mastery, and a description of their attendant conceptual difficulties."

Hiatt, A., Davis, G.K., Trujillo, C., Terry, M., French, D.P., Price, R.M., & Perez, K. E. (2013) Getting to Evo-Devo: Concepts and Challenges for Students Learning Evolutionary Developmental Biology. CBE-Life Sciences Education, 12(3), 494-508.
Abstract: To examine how well biology majors have achieved the necessary foundation in evolution, numerous studies have examined how students learn natural selection. However, no studies to date have examined how students learn developmental aspects of evolution (evo-devo). Although evo-devo plays an increasing role in undergraduate biology curricula, we find that instruction often addresses development cursorily, with most of the treatment embedded within instruction on evolution. Based on results of surveys and interviews with students, we suggest that teaching core concepts (CCs) within a framework that integrates supporting concepts (SCs) from both evolutionary and developmental biology can improve evo-devo instruction. We articulate CCs, SCs, and foundational concepts (FCs) that provide an integrative framework to help students master evo-devo concepts and to help educators address specific conceptual difficulties their students have with evo-devo. We then identify the difficulties that undergraduates have with these concepts. Most of these difficulties are of two types: those that are ubiquitous among students in all areas of biology and those that stem from an inadequate understanding of FCs from developmental, cell, and molecular biology.

A.J. Cann

Read More

Mobile phone images and video in science teaching and learning

The educational use of mobile devices has had some attention recently in terms of recording field work and also as voting devices, but considering that the majority of students now own these powerful handheld computers, it is surprising that we have not given them a more central place in education. For all out ego-centric technodeterminism, mobile technologies have had far more impact in developing countries than here. Which is why it is not surprising to see papers such as this, but how long will it be until we stop trying to control technology in education and bring it fully into play?


Mobile phone images and video in science teaching and learning. (2013) Learning, Media and Technology doi: 10.1080/17439884.2013.825628
This article reports a study into how mobile phones could be used to enhance teaching and learning in secondary school science. It describes four lessons devised by groups of Sri Lankan teachers all of which centred on the use of the mobile phone cameras rather than their communication functions. A qualitative methodological approach was used to analyse data collected from the teachers' planning, observations of the lessons and subsequent interviews with selected pupils. The results show that using images and video captured on mobile phones supported the teachers not only in bringing the outside world into the classroom but also in delivering instructions, in assessing students' learning and in correcting students' misconceptions. In these instances, the way the images from the mobile phone cameras supported students' learning is explained using a variety of approaches to understand how images support learning.

A.J. Cann

Read More

The network as proxy

A new paper in BJET discusses the use of social network analysis as a proxy for academic performance. Unfortunately, it doesn't refer to our earlier work in a this area (Badge, J. L., Saunders, N. F., & Cann, A. J. (2012). Beyond marks: new tools to visualise student engagement via social networks. Research in Learning Technology, 20).


Reviewing the differences in size, composition and structure between the personal networks of high- and low-performing students. Br J Educ Technol (1 November 2013), doi:10.1111/bjet.12110
Abstract: An interesting aspect in the current literature about learning networks is the shift of focus from the understanding of the “whole network” of a course to the examination of the “personal networks” of individual students. This line of research is relatively new, based on small-scale studies and diverse analysis techniques, which demands for more empirical research in order to contextualize the findings and to meta-analyze the research methods. The main objective of this paper is to review two research questions posed by a previous British Journal of Educational Technology contribution by Shane Dawson in order to know whether the differences in personal network composition impact on the performance of students. The two questions were defined by Dawson as follows: (1) Are there significant differences in personal network composition between high- and low-performing students? and (2) Do high-performing students have larger personal networks than their low-performing peers? In addition, the “clustered graphs” method used in this study allows the inclusion of the structural analysis of personal networks. In doing so, a new research question is addressed: (3) Are there significant differences in personal network structure between high- and low-performing students? This paper tries to answer these questions in the context of two undergraduate, inter-university and fully online courses, and two different technology-enhanced learning environments (a virtual learning environment and a personal learning environment) where interactions took place indirectly through shared resources. The results show that the network behaviors of high- and low-performing students' are strongly correlated, and that high-performing students developed larger personal networks than low performers.


The analysis in this paper (wisely) stops at correlation. But correlation is enough to get you into trouble.

"I spent 24 hours last weekend at the Isle of Wight literary festival, only in its second year but already thriving. I went to a session on spies with the writers Roger Hermiston and Professor Richard Aldrich. Aldrich has written a fascinating book about GCHQ. He said that so much of what we do now is on the electronic record that we live in a world that offers neither secrecy nor privacy. You might imagine we still have a secret ballot. Yet analysis of our buying patterns in the supermarket can, these days, reveal how we vote to a likelihood of 87%-90%." Guardian

I think it's fairly clear where learning analytics is heading, from the descriptive to the presumptive. At a time when I'm considering deciding what to buy from Sainsbury's based on a random number generator, I'm not sure what I think about these developments, except that I know I'm not sanguine about them. I believe this sort of data has real societal value, whether it be catching terrorists or supporting student learning, but the present situation we are slipping into is unacceptable. Yesterday I spent an hour interrogating Blackboard to extract data about submissions to pre-lab quizzes. I was interested in how far in advance of the practical classes students had submitted their answers, but the data buried in Blackboard tells me much more than this - not only facts such as IP addresses but far more subtle patterns. Even a simple visual inspection without software tools reveals that the majority of (but not all) students submit their answers to the quizzes multiple times. Why? Because having submitted once and seen the feedback, they then resubmit (sometimes multiple times) to ensure the mark recorded by Blackboard is as high as possible. We could infer all sorts of things from observing this behavior (such as the fact that students are engaging with the assessment part of this exercise rather than the feedback). This is the pattern followed by most students, but what about those students who only submit once, then move on - how should we respond to them? And most importantly, are these students aware of the information that their online activity patterns are revealing? Exactly how evil are the NSA, GCHQ, Blackboard?

There is a way to square the circle. Transparency. Unlike the present situation, we need to make people much more aware of what data is being collected and the implications of this. That's where the NSA went wrong. We in higher education should not follow the same path.

A.J. Cann

Read More

Grokking the visual web

I've put a fair amount of effort into getting under the skin of the visual web over the past few years. I've played with Pinterest, without success, and put this failure down to the largely non-visual nature if much of what I work on. But now that the visual web is a billion dollar trend driven by mobile devices, giving up is simply not an option.

So I'm trying again, and this time it's BuzzFeed. If you haven't encountered BuzzFeed, that's because you're old. BuzzFeed is like reddit for .... even younger people. And BuzzFeed is the hottest property online right now by some distance. When ze Frank went to BuzzFeed I didn't get it. But now I do. Yet BuzzFeed vexes me. So much trivia. And yet, when you dig a little deeper, so much potential. Strip away the froth and BuzzFeed is the place to be. Why? Because visuals drive the mobile web (typing on phone keyboards is so last year), and BuzzFeed is about visual if it's about anything. But dig a little deeper. Longform has real substance, fact checking and debunking the media, or the subtle political. Still think it's all froth? Well there's science there too. But not just science, BuzzFeed is pioneering new formats, such as reporting on scientific papers in the style of a teen romance magazine photo story. Everybody wants in. Since BuzzFeed declared "The Internet loves a list", even the Times Higher is getting in on the act.

For Open Access Week, Alun wrote 6 Discoveries You Can Read In Open Access Journals. (Yes, please up vote it!) And now it's my turn, so I've been trying to grok BuzzFeed the only way I know, by immersion:

The Wisdom of The Wire
Why The Wire (2002-2008) is the greatest TV series ever made.

You Spent Money On What? 11 Improbable Scientific Papers [NSFW]
Science is a laff. No, really, it is. Millions of pounds get spent on research into… improbable events. Here are some of my favourites.

Even such trivial immersion as this has given me the vital insight into BuzzFeed that I needed. And I like it. Why? It feels exactly like Twitter did when I first started using it, i.e. entirely trivial at first sight, but when you scratch the surface, you realize the power of the interface and what it is capable of. And the main thing is it capable of is exploiting the mobile device driven visual web. I recommend it to you.

So will I be using BuzzFeed on a regular basis. No. Here's why: I find it difficult to compose in the visual medium. I am happy to consume visual media but in terms of output I'm a words person. Something like Medium.com suits my style much better, for example, Medium's take on the visual web, or indeed, this post. Which is a shame, because Medium is rather backward looking in terms of current technologies. Or possibly, like Ev William's previous offerings, Medium is so far ahead of its time in the vanguard of the backlash against social media that we just can't see it yet.

• Creating a Buzz but is it worth it?
• Using Buzzfeed to teach history
• RSA Blogs: Why Buzzfeed is dumbing up

A.J. Cann

Read More

How to write an email newsletter

In my ongoing series of posts about email, I've been looking at different models for periodic email newsletters, and thinking about how to better connect with my various readerships.^*

For the past year I have been using weekly email newsletters on courses I am teaching. I am very happy with the model I have for doing this, both in terms of using Blogger as a content management system (CMS) and multimedia authoring tool, and the content and format of the messages. Although it is difficult to gather evidence in support of the dark social model, feedback from student questionnaires has been encouraging. I'll be writing more about that at some point in the future, but in this post I want to consider my more public facing blogs.

Last week, and after considerable thought, I announced that my longest running blog, MicrobiologyBytes, is moving to a weekly publication schedule. There are a number of reasons for this, but one of them is to allow me to avoid the information overload that comes from frequent posting, and develop a more engaging relationship with readers - something which has been lost as the number of blogs and the volume of content has grown over the years. Over the last couple of weeks I have been looking at alternative formats for email newsletters and deciding which way I want to go. But before plunging into the details, I'd like to quickly rehash the reasons behind the change. Number one is content overload and consequent loss of engagement. There's a very old AlertBox post about different categories of email which is as true today as when it was written (Protecting the User's Mailbox). This maps directly onto the problem with frequency of posting - longform once a week blog posts now jostle for attention beside short form content such as SMS and Twitter ("Protecting the User's Cognitive Space"). To protect my blood pressure, I'll reserve discussing colleagues unable to distinguish Reply from Reply To All, and Send Email To Students versus Send Email To All Course Participants, for another occasion. But this isn't an entirely negative move, I'm focusing more on the positive outcomes, such as the now near ubiquity of responsive design allowing enhanced engagement through mobile devices.

In essence, after looking at a considerable number of examples, I have concluded there are two types of email newsletter. One is the linkfest. This can be original content but is much more commonly simply a collation of previously published items. Linkfests come in two flavours, highly visual, where images serve the main come-ons to the links, or primarily text based, with some sort of branding or splash image to reassure recipients that they're not looking at spam. These are by far the most common type of newsletter. But to me, they miss a huge trick, which is the major reason I am interested in this.

Just as there are two categories of email - functional (e.g. submission receipts) and conversational - the second category of newsletter seeks to engage the readership rather than simply push more content at them. This clearly takes more work, but is the major advantage of the format. This isn't about competing directly for attention with social networks. It's about developing a much more engaged relationship with readers. There is a clear parallel with blog post style, simple sharing of a link versus writing crafted to give the impression that you are sharing the poster's inner thoughts. The newsletters I am planning to write are about relationships rather than publishing content. This arises from my experience of using newsletters with students to manage the information flow on courses I am teaching. Although much of the content I will share will consist of discussing links to content published elsewhere, the style will need to be open rather than closed, and aimed at generating conversation rather than pageviews.

Which brings us back to the issue of metrics - how do you measure success in this area? I'm coming to the conclusion the answer to that has something to do with ... Google Analytics - although it's not easy to differentiate between various sources such as email and RSS. I have a number of experiments going on right now, and I'll publish the results during my dark social workshop at SpotOn London 2013: The Dark Art of Dark Social: Email, the antisocial medium which will not die.



^*See what I did there? Used the word "series" to make it look like there is some sort of planning behind my ramblings about dark social.

A.J. Cann

Read More

Supporting assessment and feedback practice with technology

The Jisc Assessment and Feedback programme (Sept 2011-Aug 2014) has worked with 40 higher education providers seeking to review and enhance their assessment and feedback practice through the appropriate use of technology. The projects have worked directly with over 2,200 staff and more than 6,000 students covering all of the main academic disciplines and the outcomes of their work have directly impacted a far greater number (in excess of 40,000 students). This report summarises the main themes and findings from the programme up to October 2013.

Ferrell, Gill. Jisc (2013) Supporting assessment and feedback practice with technology: from tinkering to transformation.

Selected conclusions:

• Whilst difficulties in attributing specific outcomes to particular technology interventions are inevitable, there is compelling evidence that the use of technology can support the implementation of educational principles and enhance learning.
• Over-assessment and assessment bunching have a negative impact on student attainment.
• Students at all levels need better support and development to enable them to engage effectively with assessment and feedback practice.
• Students need to be made aware of the benefits of engaging with assessment and feedback practice and developing their own evaluative capabilities in terms of improving their personal employability.
• End-to-end electronic assessment management is not easy to achieve but, for possibly the first time, the technical capabilities exist if institutions can design and implement appropriate business processes. (Hmmm...)
• There is a compelling case for e-submission in terms of improving efficiency and meeting student expectations.
• Online marking has reached a level of maturity whereby the evidence for its effectiveness and efficiency may soon result in a critical mass of institutions and academics adopting the practice.
• Electronic assessment management is a pre-requisite for capturing and analysing the data needed to enhance learning through effective use of learning analytics.

A.J. Cann

Read More

TiL ... I have zero prospect of getting to the end of this FutureLearn MOOC

Left some feedback on the site - which is very interesting as it allows you to see the feedback left by other people. This confirmed my feelings about the FutureLearn platform - it's barely a beta at present. I did get an email response to the comment I left.

And linking course units to articles behind commercial paywalls is ... perverse. I'm not convinced these guys get "MOOC".

A.J. Cann

Read More

Pinterest as a learning tool?

I have tried and failed with Pinterest, but if it's your thing, you might be interested in this:

Learning beyond the classroom: evaluating the use of Pinterest in learning and teaching in an introductory anthropology class. JIME http://jime.open.ac.uk/2013/12
Abstract: This paper details a case study of using Pinterest as an educational resource in an introductory anthropology course. Its use was evaluated through the data provided by the platform itself and focus groups. This evaluation found that Pinterest was a popular and useful tool for developing curated multimedia resources to support students' learning. The focus group findings suggested that online resources were shared by students across a variety of social networks, including but not limited to Pinterest. These resources were shared and used beyond the classroom, both physically in locations outside, but also with friends and family that were not part of the classroom. The opportunities for developing critical thinking through the use of tools such as Pinterest are explored.

A.J. Cann

Read More

Blog email subscription options #solo13

The Dark Art of Dark Social: Email, the antisocial medium which will not die.

Follow Science of the Invisible by email
Enter your email address:


Email direct to your inbox - the easy way to read this site.

Email newsletters go back to the dawn of the Internet but have been neglected in recent years with the rise of shiny social networks. But eTOCs have seen off RSS and email will not die. Why else does Twitter encourage users to share tweets via email? The evidence is clear that message dissemination via social networks only reaches a relatively small proportion of the potential audience and that most link sharing occurs via private channels such as email (hence "dark social"). The Silk Road bust and the unmasking of Dread Pirate Roberts reveals how difficult it is to maintain anonymity online. Public discussion via online comments is limited by the high risk nature of the putting your head above the parapet in the rough and tumble of the web. Yet discussions via private channels continue unabated. The rebirth email on mobile devices and the contribution of social media to information overload is causing a rethink about the utility of public versus private channels and we are seeing the rebirth of email newsletters and mailing lists. My upcoming workshop at SpotOn London 2013 will look at the case for and against dark social, practical aspects such and tools such as how to use blogs as content management systems for email generation and asks you to embrace the dark side. As part of my research I have been experimenting with email subscription options on my various blogs (see DarkSocial Experimental Ethics). So what are the options?

MailChimp is what the many of the big boys use. Very powerful and free for up to 2000 subscribers (after that, you pay). Downsides: slightly dodgy stats claims, doesn't play well with Microsoft Outlook, can be tricky to get your posts formatted the way you want. The good news is that MailChimp emails allow replies, offering up the possibility of onging email conversations (if that's what you want).

Feedburner offers an email subscription service. Available for any platform and particularly easy to integrate with Blogger via a sidebar widget. Unlike MailChimp, recipients cannot reply by email but they can leave a public comment on the original post. Like MailChimp, Feedburner allows email replies.
Key feature - unlike the other two solutions listed here, you can burn a selected tag as a feed for email subscriptions rather than inflicting all your verbal blogging diarohoea on subscribers inboxes. This enables you to schedule a regular newsletter rather than turning them off by bombarding them with everything you post. Blogs are great for those who have the time and the commitment for frequent posts. But many don't and not everyone wants to read your passing thoughts as they occur to you. The answer for both of these groups is the digest or newsletter - weekly, fortnightly or monthly as you prefer. So how do you structure and write an effective email newsletter? If enough people ask me in the comments to this post, I'll tell you ;-)
But there's a problem hanging over Feedburner - the future status of the service after Google axed Reader earlier this year. Feedburner's stats and post scheduling also seem to be as erratic as ever :-(

Wordpress JetPack also offers an email subscription option (run via wordpress.com even on self-hosted installations) but unlike the other two is only available for Wordpress blogs (on wordpress.com or self-hosted). Probably the simplest of the three to set up. Like Feedburner email subscriptions, simple to run - the blog is your content management system delivering full text content or excepts to your subscribers without any work after you have scheduled or posted them. Unlike the other two services, JetPack emails do not offer email replies, so if you think email newsletters are about relationships rather than publishing content (hint), you'll have to include your email address if you want people to be able to contact you.


So why bother with email at all? Long answer - because it works (Why email newsletters still work). Short answer: mobiles.

So am I going to sign up to your blog via email? Well no I'm not, but I'm unusual - a web dinosaur who's still in love with RSS (the glue of my online world). But I'm in an ever decreasing minority in that respect, and the advantages of email on mobile devices are considerable.

A.J. Cann

Read More

One day all journals will be like this

Research in Learning Technology introduces article level metrics. Such as:

A.J. Cann

Read More

And we're off #FutureLearn

After a quick Chatham House Rules gander at the FutureLearn platform last week, I'm up and running with my first MOOC, The Secret Power of Brands.

"Over the next ten weeks, we’ll be exploring our topic step by step, using a variety of tools. There are opportunities to test yourself along the way, to be sure it’s all sinking in, as you learn about the basics of branding. You'll have a chance to use your own creativity. You'll analyse how organisations use their brands. And you'll be assessing the future – both the future of branding and the future of you."

While I am interested in branding, overall I'm less highly motivated by the subject than I am about plant science, so this will be a fairly stiff test for the stickiness of the platform. Tbh I though there was a nice symmetrical irony about this topic too :-) 10 weeks? We'll see...

The spare look and feel of the platform is very iOS7 - Jacob Nielsen might well have things to say about the design. It is definitely lacking in navigation cues, something which needs fixing urgently. The experience was a bit laggy for me too, they're going to have to spend some money on bandwidth. Chunking is a bit uneven, most of the videos a comfortable 2-5 minutes then suddenly an 18 minute chunk of lecture notes (© 2009) on the history of brands.

Assessment in this course so far is 5 simple MCQs at the end of the unit, although with nice feedback for failed attempts.


Verdict so far:  The jury is out.

A.J. Cann

Read More

#DarkSocial Experimental Ethics #solo13

I'm trying to design some experiments to look at the impact of dark social, but I want the ethics of my approach to be sunshine and light.


I've signed up some interesting contributors for my workshop on DarkSocial at SpotOn London 2013 but I still feel I would like to try to generate some data so that my contribution is not purely anecdotal. I have analytics stats I can show of course, but l last week I was fretting about how I could demonstrate that it's not all about numbers and that dark social channels lead to high levels of engagement. If I ask students to complete an online survey via email I get low response rates, but survey responses continue to nose dive, so that's not surprising. Around half the email conversations I get from students on courses I am teaching originate from mass emails such as my weekly newsletters, but that's a fairly small scale experiment. I'd really like to step this up a gear and draw some wider conclusions.

After investigation, it's clear that MailChimp, ReadNotify, etc, can't actually deliver what I need in terms of verifying engagement. It's not their fault, modern email systems are designed to maintain privacy, which is a major reason why people like them so much. So my only alternative seems to be trying to measure what people do in response to email rather than going down the route of simple numbers. Which is where the problems start. To their credit, MailChimp and ReadNotify make it very clear that this is murky territory and that the worse thing you can do is cause loss of trust by intruding too far. So I need to find ethical ways of measuring these relationships, and here's my framework for doing that:

1. Any task I ask email correspondents to perform in order to measure response rates has to be authentic, i.e. something that that might normally do.
2. Any task-based measurement has to offer the recipients a potential benefit for participating, for example, links to extra information or service that they want.
3. And?

I think I'm happy with this. All I need to do now is figure out what I can offer (calls for a sand walk - when it stops raining :-).
Lou Woodley has just pointed out (via a dark social channel, of course :-) that this may be one reason that dark social usually sits within the domain of the Marketing department rather than rather than traditional analytics. And that's a pretty good reason for wrestling it back off them :-)

A.J. Cann

Read More

One week on... [Pre-lab Quizzes]

... and I have the data from the second online pre-lab quiz which I wrote about on the School blog last week.

This week, out of 190 students registered on Blackboard, 64% (87%) submitted answers before the practical, 7% (5%) looked at the quiz but did not submit before the practical and 8% (8%) did not look at the quiz online before the class (c.f. figures in brackets for the first week). As previously, a considerable number of "just in time" submissions were returned.

There were some really insightful answers submitted citing references - clear evidence of independent reading in preparation for the practical. The on time submission rate dropped, but the big change this week was the number of students submitting blank answers to the essay format questions, presumably in order to see the feedback - up from less than 1% in the first week to 21% in Week 2. Many of these students are clearly testing for weaknesses in "the system". We've already cut this one off at the pass by switching from Essay format questions to Fill-In-The-Blank format, as I explained earlier in the week (TiL - a better way of using Blackboard) - so this strategy won't work next week as by automarking FITB questions using pattern matching, such students won't see the feedback but will be told to answer all the questions.The first definitive conclusion from this pilot study it therefore that essay format question are not suitable for these quizzes. FITB questions will certainly reduce staff input required but whether that are an improvement in this circumstance remains to be tested.


Thus the never ending war of wits between fresher and grizzled lecturer continues.

A.J. Cann

Read More

What a Plant Knows [MOOC]

What a Plant Knows comes as a refreshing change. This is down to the quality and enthusiasm of the teaching staff rather than any platform attribute.

Apart from a couple of statistics courses, the majority of the MOOCs I have taken were because I wanted to explore the platform and approach to learning being used rather than because of the subject matter. Coursera's What a Plant Knows is different, because as the non-plant scientist Internet Consulting Editor of Annals of Botany, I feel that I really do want to learn more about plants.

Based on his book What a Plant Knows, Daniel Chamovitz fits into what I'll call the Model B MOOC Professor - the big personality. In the grey world of MOOCs, this works well for me, although it would be very easy to tip over the edge and become irritating. As usual, there is a little too much talking head video, but clearly efforts have been made to include alternative formats. The assessment component is perfunctory, a few MCQs for each section. To their credit, teaching staff, including Daniel Chamovitz, are actively participating in the course discussions boards.

Week 1 was a good general introduction, although maybe slightly a little too "OH WOW, it's a PLANT". Week 2 on plant responses to light ("What A Plant Sees") is right on the money - great stuff! Without any doubt this is the best Science MOOC I have seen yet.

Will this (very good) MOOC bring students flocking to the professional study of plant science? Not in any significant numbers - I can't see us having to start a plant science degree to cope with student demand any time soon.

A.J. Cann

Read More

Negotiating learning patterns through little big data

Since last week I've had over 150 emails from students on the two first year modules I am teaching. I am aware that this is a nightmare scenario for many people (i.e. the combination of teaching and email), but it's exactly what I've been trying to achieve since switching from an open social to a dark social communication strategy a year ago. As expected, at least half of the messages have come from smartphones.

With a head count of approximately 500 students on the two modules, these emails amount to something like a 30% response rate, 30 times the response rate received via social channels such as friendfeed or Google+. They have come in response to verbal invitations in lectures, online help documentation and the weekly email newsletters I publish (via BCC) to students on the courses I am teaching. Of course, the use of email does not preclude face to face meetings for students with greater needs, providing a sifting mechanism for needs assessment. I am not using email to deliver teaching, only to support learning and the inevitable administration involved.

Although deliberately encouraging this volume of email is regard as lunacy by many, I find the killer attributes of email (ownership of your inbox and the convenience of everything in one place) makes it quite easy to manage. I do use canned responses (Postbox Drafts and Templates and a few macros) but these are overlaid with personalization (e.g. use of student first names) which is very important. Students feel that they have direct staff contact via the phone in their pocket or their treasured new laptops. I feel that I am in control of my communications strategy without madly checking many different social networks. Manageability is enhanced by training and management of expectations - students learn that email is not real time conversation through me not replying to messages until an hour after they have been received, and any student who shows signs of becoming over dependent is gently weaned by slowly increasing the response time.

Is this the best use of my time? It is hugely more effective in terms of response rate that the open social strategy I followed in previous years (Visitors and Residents: mapping student attitudes to academic use of social networks) and it is much more popular with students because of the personalization element - I am talking directly to them, not broadcasting. Email results in far less inhibition than public sharing of information, hence the much better response rate and hugely increased student satisfaction.

But this strategy really only pays off if I use the data collected from this channel to focus my efforts and attention. By grouping the requests and questions into categories, I can prioritize the information I feed back and can quickly spot and fix any bottlenecks or problems. By using this nano-scale "big data" in this way, the effort more than pays off. But from the perspective of a fresher taking their wobbly first steps in higher education, I am there for them via the device in their pocket when they need me.

Now in case you thought this post was just another of my ramblings about dark social, we can get to the point - what I have learned from the data I have collected in the last week. By far the most frequent category are the calls from students for practice assessments and resubmission of assessed work. Since these are not our practice, it is clear that we need to manage student expectations in this area. The majority of students arrive from secondary education with the experience of mark manipulation through resubmission and with this as their most important feedback channel. We clearly need to do a better job of explaining that independent learning in higher education is not managed by this route and they they need to pick up on the more limited formative feedback they will receive. We can do that by emphasizing the significance of independence in higher education and contrasting this approach with their prior experiences. And if that's not a good use of email, I don't know what is.

A.J. Cann

Read More

EduWiki Conference 2013

Last year I went to the EduWiki conference and I was very impressed.

This year's conference is in Cardiff Friday 1 - Saturday 2 November 2013.

If you're in that area, I'd recommend it.

A.J. Cann

Read More

TiL - a better way of using Blackboard

File this under the heading Blindingly Obvious, but if it stops someone else making the same mistake, it's worth sharing.

This year we are piloting online pre-lab quizzes via Blackboard to increase student engagement with laboratory practical classes. The quizzes are formative and we are interested in student response rates, as well as delivering useful feedback to students (including directed reading) to improve their knowledge before they take the practical class. So far, the first results look promising.

The quizzes are deliberately designed to be challenging, and to make students read and think about the science they will be doing in the practical class rather than blindly following instructions like a recipe book. For this reason we have been using a range of question formats, including MCQs and fill-in-the-blanks questions. However, to maximize engagement we also want students to write some short answers of about 100 words explaining their reasoning. Initially, I used Essay format questions for this, but they have a serious limitation in our situation. We have hundreds of students taking these modules and we want to give them feedback quickly, and to have a "gamification" element derived from a formative mark. But Blackboard Essay format questions require manual marking. Even though we are only interested in delivering the feedback to students so they can compare it with their answer, manual marking is slow and tedious and removes the instant score element which we know many students respond to.

So I have switched the Essay format questions to Fill-in-the-blank format. This doesn't look as good on screen as students are only presented with a small text entry box (which there is no way of formatting in Blackboard as far as I can see) rather than a proper paragraph submission tool, but we tell them to paste their answer in the box. The big advantage comes in automated marking. By selecting the Pattern Matching option with the correct answer set as ".*", students receive the overall mark for the quiz instantly without any staff input. If they do not answer the question, they don't get the feedback but are politely asked to submit an answer.

It's not the prefect solution but it is an improvement on the first iteration, and we'll see what the students say about it at the end of the module.

A.J. Cann

Read More

Tagginganna for science

What seems like years ago (because it was), some of my colleagues in Leicester started the Tagginganna project, involving inline commenting of texts. I'm not quite sure what happened to Tagginganna eventually (is it still in use?), but I do know that the concept of inline commenting - one of the "power features" of Google Docs for example - is still grossly under used.

New formats for scientific publishing such as PLOS (and latterly even Nature) have adopted online commenting as a new standard for scientific publishing. But online commenting on papers has never taken off - most papers languish without discussion in the comment section. For that reason, the launch of PeerJ Questions is particularly interesting. Although I like inline commenting, there's no particular reason to think that it would promote the frequency of comments much above burying them at the end of the paper or on another page, so PeerJ Questions uses a Quora/Digg/reddit-like voting system to try to surface the most "popular" questions.

But are questions about published manuscripts ever "popular"? In my experience most authors want the published paper to be an archival file-and-forget experience, with public discussion about as welcome as a fart in a spacesuit. Which is not to say that post-publication discussion does not go on, it simply occurs via the dark social channels it has always occupied.

You could argue that it is the high risk nature of sticking your head above the parapet in science that inhibits commenting (and is also the reason why science clings fiercely to anonymous non-transparent peer review). But if you're slightly more optimistic about the human condition you might hope to believe that comment voting will increase scientific discourse by lowering the barriers to contributions. It is no surprise to me that this innovation has come from PeerJ. If it is successful you can expect others to copy it just as they copied PLOS comments.

A.J. Cann

Read More

Digital literacies concept is not helpful

This week, when I've not been stressing over the first year timetable, I've mostly been stressing about digital literacies.

As our student Digital Literacies in the Disciplines (BioSET) project cranks back into life (the private Facebook group did not sustain activity over the summer break), I've been thinking about the barriers we face. Participating students are obviously confused - the digilit concept is confusing in its own right, and the employability layer on top is a step too far. We are resolving the confusion by stepping back from the student-led approach we wanted to follow and giving the participants more direction about activities, at least to get them started.

But I've also been thinking about the fundamental problem with the digilit concept. "Digital literacy" is either completely transparent (therefore impossible to grasp) or merely an artificial construct with no real world value. I think it's both!

Are our student participants digitally literate? Clearly yes.
Can they articulate this literacy? Not easily.
Why not? Probably a number of reasons, but the lack of utility of the digilit label is a contributing factor.

And Xerte? It's a problem we don't need, so it's on the back burner for now.


Contributor blog posts so far:

Student Employability Team

Logo Design

Evolution of a Biology Logo

BioSET: Digital Literacies Project Introduction

First Meeting of the New Academic Year!

- more to come soon hopefully, following our project meeting today.



Dean Groom: Digital Literacy is a crock

A.J. Cann

Read More