What You Need to Know About Learning Analytics

This special report caught my attention partly because our workshop with Diana David when she showed us how to access BTC's learning analytics. This special report is from The Chronicle of Higher Education. It won't be available online for 30 days, but I'll leave a print copy of it in the eLearning Lab. (It has some serious copyright information/threats in it. I know that The Chronicle can charge up to $90.00 for a 24 page special report.) This report is a reprint of a series of their best articles on the subject. I will highlight a few of them.


Jeffery Young's This Chart Shows the Promise and Limits of 'Learning Analytics' (2016), reflects on what he learned from "the spider graphic" created from one his online courses. The graphic shows the student activity. This instructor, Courtney Stewart, has seven years of online teaching experience. With each of his lessons he presents the material in many different formats--PowerPoint, Podcast, video, and straight text. He was expecting to see the most popular formats or what format worked well for what percentage of students. He discovered that over half of his students never used the Home page or the lessons, but would jump straight to the homework. He decided to attach the lesson material to the homework posts. Students would then have only one place to look. The instructor included these analytics in his tenure portfolio.


The article that I thought would interest me most was Michael Feldstein's Reaching Students in the Back Row (2016). He discusses the three methods that he and his colleague, Phil Hill, have discovered that can help the student who is struggling or disinterested. 1) Move content broadcast out to the classroom. Online this means finding alternate ways to present the information--videos, podcasts, etc. 2) Make homework time contact time. Don't use robo-grading all of the time. Personalize some of it. Make the homework relevant to what the student it learning or will use later in the class or in their profession. 3) Hire a tutor. Sometimes machine-graded homework can also work as a robo-tutor. These are supposed to personalize learning. This article disappointed me. The title promised more than it delivered.


I discovered in this next article, Adaptive Learning Earns an Incomplete (Feldstein, M., 2016), that I did not totally understand the meaning of adaptive learning. I thought it was more accessible learning. It is a computerized what of doing what teachers have always been doing, noticing that certain students are not understanding the material and adapting the instruction to include different ways to teach the same thing. The research was funded by the Bill & Melinda Gates Foundation. Out of 23 courses, only four of them showed a modest gain in grades; most had "no discernible impact." The author points out some limitations to how the data was created and interpreted.


The last article, also by Michael Feldstein, Understand the Origins of Ed-Tech Snake Oil,  has implications beyond educational research to today's current affairs. It discusses how companies need to be made accountable for the claims they make about the effectiveness of their product. It details a report presented by an educational software product that they claims to increase graduation rates. There was no peer-review, but since it was presented and reported at many educational conference, people start quoting the statistics. Then a critic of the report questioned the findings and thought there were statistical errors due to poor design. One critic finds that feeding students chocolate can produce the same effect. The company's designers are asked to present additional information, the company chooses not to comment. It is a good example of how snake oil or what we call false news is spread. The author argues for better research design and peer-review, but to invite vendors to be part of the peer-reviewed research.


It would be interesting to see what analytics Canvas can produce and how instructors can access them. It is important not to determine if the analytics are worth the time it would take to create them. Young concluded that often looking at the straight data can be as valuable or more valuable than data presented in a labor-intensive graphical form.

Learner Attributes & Thinking About Our Student Focus Groups

"Instead of looking at the attributes of learners who eventually drop out to determine how to help learners sustain motivation and meet learning goals, discovering the attributes of successful learners can help us understand factors that contribute to motivation and persistence. Attributes that successful learners have include a high level of self-confidence and self-efficacy, and an internal locus of control."

This quotation from a book I'm reading with a few other Quality Matters Coordinators called "Effective Online Teaching", by Tina Stavredes, recalled for me our recent FLC conversation about our upcoming Student Focus Groups. I think it will be important to hear from students who struggle with online learning as well as from those who are more successful.

Self-confidence: "individuals' belief in themselves and their ability to succeed in general"

Here's an interesting and brief post from Cengage "Three Factors that Boost College Students' Confidence"

Self-efficacy: "person's belief that he or she can succeed at a specific task or range of tasks in a given domain"

Here's a good scholarly article called "Academic Self-Efficacy and First-Year College Student Performance and Adjustment"

Locus of Control: "beliefs about what determines...successes or failures in life"

Here's a Khan Academy video called "Locus of control, learned helplessness, and the tyranny of choice" (not a fan of animal research, btw)

So, what kinds of questions might draw out information about our focus group participants' self-confidence, self-efficacy, and locus of control? 

To BBB or not to BBB, that is the question!

Today was the first time FLC held its virtual meeting. We used the Big Blue Button (BBB), a pretty cool conference tool on Canvas… if you know how to use it. Thanks to Marcia and Dawn who helped us navigate the tool. Among the participants were Mary, Alyssa, Janell, and Vicky. It was hilarious at the beginning when we found out that each of us could be the moderator, could assign someone else to be the moderator, mute and unmute the other participants, and then once in a while we heard Janell slurping her coffee… and poor Mary who could only type her conversations on the side and inadvertently got ignored, because we were mostly looking at our own images or the screen in the center and so we completely forgot to look at the chat off the side (until we turned on the audio chat notification).

One of the big drawbacks of BBB is that it only works on PC and not on mobile devices. Jane tried it on her phone at the airport and I tried it on my iPad, and it just didn't work. Too bad. So we were using each of our personal computers at the comfort of our home or office trying out this “new” tool (at least to most of us). Another drawback according to Marcia is that BBB apparently can only host up to six participants at a time. So she suggested another conference tool called “Go To Meeting” which perhaps we can try next time. For math instructors like Vicky and me, we found BBB is quite clunky…. can’t write mathematical expressions too well with a mouse or it will require us to use one of the tablet computers and bear with the time lagging. Dawn also added another drawback, i.e. it can only keep the recording of the conferences for 14 days. Check out the recording of this first virtual meeting before it disappears in 14 days! (By the way, we forgot to start the recording earlier, so you will only see a partial meeting there.)

Anyway, we accomplished a lot actually in two hours going over the items on the agenda for the day. Dawn introduced another tool called “Padlet,” a brainstorming virtual board. So we also monkeyed around with it, trying to post a few messages and pictures here and there. Then we talked about the recent QM workshop that a few of us participated. It’s called “Using Instructional Materials and Technology to Promote Learner Engagement” focusing on QM Standards 4, 5, and 6. While we found the overall experience is valuable, it is ironic that we the participants didn't find the short course as engaging as the title suggested.

The conversations continued to touch issues on whether we could “require” online students to participate in virtual conferences as a way to engage students. We were wondering whether there is an existing document of online learning expectations, policies, minimum requirements, etc. If not, perhaps FLC can initiate the formation of such document and involve administration to get on board with it. Marcia started a discussion on this on Canvas discussion boards. I’d love everyone of us to chime in. I think our collaborative efforts will help tremendously in dealing with some of the frustrations when students do not respond as we expect them to.

Toward the end of our meeting, I expressed an interest in having another virtual meeting using BBB (or any other tool that we decide to try). Vicky brought up a good point that practicing more with these technology tools (conferencing, discussion boards, etc.) would increase our confidence in using them with our respective students. I think Faculty Learning Community is just the perfect place for us to do that! Before we all left the conference room, Marcia left us with this note: "If you want a chuckle after we are finished, watch this YouTube on web conferencing http://www.youtube.com/watch?v=DYu_bGbZiiQ." I don't want to be the spoiler. You've got to watch it yourself.

On a personal note, I thank all of my colleagues for their unique personalities, talents, and expertise. I am thankful for FLC, from which collegial community I gain invaluable knowledge, skills, and overall learning experience as a way to help me to be a more effective instructor teaching online classes that could be challenging at times.

Happy Thanksgiving all!

Anita

Meeting # 3 - What Does Online Community Look and Feel Like?

For meeting # 3, our group decided to use the time we'd carved out for today to work on the Quality Matters online workshop that many of us are taking, which is called Using Instructional Materials and Technology to Promote Learner Engagement.  We set up refreshments in the eLearning Lab, where some of us decided to work in order to avoid the typical interruptions we experience in our usual offices and work areas. I loved seeing and hearing our group members drop in and/or settle in in the lab, and I also loved seeing that so many of us were online working on the course, too (look at the "Participants" list under "Navigation" in the QM course). We are still forming as a group, but despite all of our different physical locations, I felt like part of a community today! Isn't this exactly what we are thinking about when we consider engagement and community in online learning environments?

As I searched for engaging learning materials as part of an assignment for Module 3 in our QM course,  I came across a video that I really liked. It met the objectives for my course assignment, and I think it's appropriate for our FLC, too! For context, I will share my assignment responses (which are also posted inside the QM course) along with the video (link is also posted below). Enjoy!

 

 

Dawn's QM Assignment:

Through MERLOT II, I found an excellent video called "Engaging and motivating students". Access the video here: [http://online.cofa.unsw.edu.au/learning-to-teach-online/ltto-episodes?view=video&video=221]

How does your selected material align with your objectives or competencies (please post the objectives or competencies with which your selected material aligns)?

Course Objective: “Upon completion of the course, Learners will be able to define student engagement in the context of online learning.”

Module/Unit Objective: “Learners will describe the definition of student engagement in presentations to their colleagues”

This video is part of workshop and training material from Learning to Teach Online at the University of New South Wales. It addresses the topic and nature of student engagement online from the perspective of educators in a presentation format, all of which supports and aligns with my learning objective.

In what way does your selected material meet Specific Review Standard 4.4, "The instructional materials are current" (refer to the Annotation for Specific Review Standard 4.4 for more information on what it means to be "current")?

This material is posted on a current and maintained university website and “represents up-to-date thinking and practice in the discipline” as described in the annotations for GS4, SRS 4.4.

In what way does your selected material add variety to your course in terms of the Annotation for Specific Review Standard 4.5?

This material is in captioned video format, which will provide variety from the text and other printed readings. Students will be able to access the content using captions, if they prefer. Also included with this video is a downloadable pdf of Strategies and Tips for teaching online.

How would you cite your selected material in order to meet Specific Review Standard 4.3?

McIntyre, S. (2011, January 19). Engaging and motivating students. Learning to teach online. Video retrieved from http://online.cofa.unsw.edu.au/learning-to-teach-online/

 

Engaging Discussions Webinar - #sharemynotes

On Tuesday, 10/11/16, Judi, Traci, Jane, Alyssa, Anita & I gathered in the eLab to watch a QM sponsored webinar on designing engaging discussion boards. I thought I'd share my notes, which are really just an incomplete list of ideas (I am sort of notorious for forgetting to take notes during meetings). But I think there may be some useful nuggets for our group.

Anyone who attended, please add your thoughts in the comments (and fill in any of my many gaps).

A few points about students and online discussions

• Students can feel very isolated in online environments, and online discussions can help bring students together. 
• Students learn from each other.
• Discussion boards are great places for students to practice thinking, writing and connecting.

 Ideas to make online discussion engaging

• Have a catchy title (this could go too far in some circumstances, but I thought of one of Anne Lamott's chapters in Bird by Bird-- used by one of our own faculty, and available in our own BTC Library-- "Shitty First Drafts"); ask a provocative question in the title.
• Invite a guest speaker, then facilitate conversation via the discussion board.
• Ask open-ended questions (here are a few OEQs, what do you think of them?).
• Ensure you (the teacher) have a presence on the boards -- chime in, ask guiding questions, suggest areas of inquiry, etc.
• Make sure to align discussion with other course materials and activities; adult students want to see connections between what they are learning and what they are doing (andragogy), they also want to apply their learning to real life.
• Discuss arguable topics - controversy inspires interest; this also presents an opportunity for students to discuss and practice online etiquette and civility (in a format that you the teacher can monitor and influence).
• Students respond to authenticity-- be yourself and consider writing more like how you talk when you set up discussions and when you jump in to coach or validate what students are saying.
• Embed videos (your own, YouTube, etc.) in the discussion board and ask for responses -- this is engaging for several reasons, including that it reduces the clicking around that students have to do in order to access the discussion content and instructions.
• Use scenarios-- present a situation in the discussion, then ask the students to predict what will happen (bantering is good).
• Use audio - talk to them (one of my professors in grad school always posted an audio version of her notes on the upcoming week's objectives and activities along with her written notes. It was a huge relief to be able to absorb information withOUT reading, and also very nice to hear her actual human voice in that challenging online class).
• Be modern (use memes, cartoons).
• Use real graphics & photos, especially of people-- always look for ways to humanize the online environment. (Also, remember to give credit-- we have a built-in awesome Attribution Builder now...)
• Use bullets & organize your instructions with headings (allows them to scan and is accessible).
• Ask yourself, will this discussion connect students to each other?

 

Have a happy day! Dawn

Our First Meeting

I felt nervous as the clock ticked its way to 2 pm. Who would show? Who wouldn't? I checked the chaffing dish of assorted hot vegetables-- tempura battered mushrooms, broccoli and zucchini, baked sweet potato wedges-- that Annie from Culinary Services had just delivered for us. I poured myself a small glass of ice water. In my mind, I ran through the list of people I'd invited to our gathering, taking the time to call up each person's face (a habit I've cultivated to humanize "likes" on social media), trying to imagine how they might be experiencing a Fall quarter Friday afternoon. I suspected some of us would be feeling more enthusiastic about an end of the day, end of the week meeting than others.

Alyssa and Mary from our Accessibility Department were the first to arrive. Alyssa, a calm, friendly person who will always laugh at your jokes, was happy to see the veggies. Mary, a warm and communicative student advocate and bridge-builder, investigated the identities of the mystery sauces that Annie brought for us. 

I thanked them for coming.

Marcia and Janell, two of my favorite BCIS faculty from the floor below, came in. Marcia, a truth-teller, shared a conversation she'd just had with another employee questioning the thinking of anyone who'd plan a meeting on a Friday afternoon. Janell, who is kind and makes excellent use of time, just smiled very broadly at me, so I knew she had some thoughts about Fridays, too. 

I thanked everybody for coming again.

Judi came in. I could tell she was thinking about something, she's always thinking, planning and working things out. A positive person, she says "yes" a lot. She tries new things, lots of new things. We follow each other on several social media sites, so I know she travels and hikes and lives with a beautiful dire wolf named Shiya. 

We all gathered around the tables in the eLearning Lab and chatted for ten or fifteen minutes, and then I thanked everyone again for coming. I pulled up the Canvas site that I'd structured for the FLC to use and shared the agenda for our gathering, explaining that I hoped to discuss the main points of our grant, give them a tour of the Canvas site, share a few resources, and pick some dates for future gatherings. 

As we were looking at the over-arching questions from our grant documentation, ("What do we mean by student engagement, and how does it contribute to course completion and success in online environments? What characteristics of online course design, learning materials, activities, and technologies are most likely to create or discourage student engagement? What are best practices for creating accessible online learning materials and activities using technology?"), Jane joined us. She is the busy director of Library and Media Services, very wise, and laden with institutional knowledge. She is also my former supervisor of many years. I am certain we could finish many of each other's sentences without much effort.

I remembered that we hadn't yet done our group activity, Expectations: 

• Tell us your name and describe your role here at BTC.
• What do you expect from this faculty learning community?
• Please make a wild prediction of the best possible outcome if your expectations for this group come true-- be specific-- and as outrageous as you like!

Here are the word clouds of our responses.

Here is the word cloud of all of our posts merged together:

Anita, our newest tenure track math teacher, who is efficient, precise, and working hard to figure out how our institution works, joined us after her last class, as the group discussed an upcoming webinar on designing engaging online discussions. I was very happy to see her. I think she will gain a lot from spending time with other faculty, and I think the group will benefit from her naturally great observations, as well as from the fresh perspectives she brings as a new employee.

Most of us are planning to register for an online QM class, Using Instructional Materials and Activities to Promote Learner Engagement, which begins on November 3rd. I'm looking forward to focusing on standards 4, 5 & 6, which are foundational to our FLC topic:

"how course design, materials, learner activities and course technologies affect student engagement and student success in online learning environments, using QM Standards 4 (Instructional Materials), 5 (Course Activities and Learner Interaction), 6 (Course Technology) and 8 (Usability and Accessibility) to frame our inquiry."

I feel very lucky to be working with this amazing group of people to build a community and to figure out how to make online learning better for our students.