I have written (or am in the process of writing) several chapters on web analytics in education. The following is a list of some of these readings about Web Analytics. Feel free to comment and share other helpful sources.
Web Analytics References
Ballardvale Research (2007). Market Trends – Web Analytics: History and Future. Online document. Retrieved 8/24/2007 from http://www.ballardvale.com/free/WAHistory.htm
Burby, J., & Atchison, S. (2007). Actionable Web Analytics: Using data to make smart business decisions. Web Publishing, Inc. Indianapolis: Indiana.
Cadez, I., Heckerman, D., Meek, C., Smyth, P., & White, S. (2003). Model-based clustering and visualization of navigation patterns on a Web site. Data Mining and Knowledge Discovery, 7, 399-424.
Comunale, C. L., Sexton, T. R., & Voss, D. J. P. (2001-2002). The effectiveness of course Web sites in higher education: An exploratory study. Journal of Educational Technology Systems, 30(2), 171-190.
Gao, T., & Lehman, J. D. (2003). The effects of different levels of interaction on the achievement and motivational perceptions of college students in a Web-based learning environment. Journal of Interactive Learning Research, 14(4), 367-386.
Hellwege, J., Gleadow, A., & McNaught, C. (1996). Paperless lectures on the Web: An evaluation of the educational outcomes of teaching Geology using the Web. Proceedings of the Thirteenth Annual Conference of the Australian Society for Computers in Learning in Tertiary education (ASCILITE ’96), Adelaide, South Australia, Australia, 289-299.
Horváth, A., & Jókai, E. (2007). Evaluation of University E-learning Courses Based on Data Mining Methods. Presented at the 6th European Distance Education Network (EDEN) Conference. Stockholm, Sweden.
Hwang, W.-Y., & Li, C.-C. (2002). What the user log shows based on learning time distribution. Journal of Computer Assisted Learning, 18, 232-236.
McIsaac, M. S., Blolcher, J. M., Mahes, V., & Vrasidas, C. (1999). Student and teacher perceptions of interaction in online computer-mediated communication. Educational Media International, 36(2), 121-131.
Nachmias, R., & Segev, L. (2003). Students’ use of content in Web-supported academic courses. The Internet and Higher Education, 6, 145-157.
Peled, A., & Rashty, D. (1999). Logging for success: Advancing the use of WWW logs to improve computer mediated distance learning. Journal of Educational Computing Research, 21(4), 413-431.
Rafaeli, S., & Ravid, G. (1997). Online, Web-based learning environment for an information systems course: Access logs, linearity and performance. Proceedings of the Information Systems Education conference (ISECON ’97), Orlando, Florida, USA, 92-99.
Rieger, R. H., & Sturgill, A. (1999). Evaluating Online Environments: Tools for observing users and gathering feedback. New Direction for Evaluation, Number 84, 45-58.
Rogers, P.C., Flores, D., Matthews, K. (2007, Sept). The Learner/Teacher, Online Platforms, and Web Analytics in Learning Design. Presented at the Open Education 2007: Localizing and Learning, Utah State, Logan, Utah.
SenGupta, S., Hopson, R., & Thompson-Robinson, M. (2004). Cultural competence in evaluation: An overview. New Direction for Evaluation, Number 102, 5-19.
Sheard, J. I. (2007). An Investigation of Student Behaviour in Web-based Learning Environments. A dissertation accepted by the Faculty of Information Technology at Monash University, Australia.
Watt, J. H. (1999). Internet Systems for Evaluation Research. New Direction for Evaluation, Number 84, 23-43.
Zaiane O., Han J. (2001) Mining for E-Learning Gold. Retrieved from http://wildcat.iat.sfu.ca/theme3/Zaiane1.html
Several months ago, before I knew I was going to be directing the Edulink project, I proposed an idea to Andy Gibbons, the chair of the Instructional Psychology and Technology department at BYU, to offer a course on the use and application of web analytics for designing and improving online learning environments, and he loved the idea.
I had been teaching a course on Web Analytics in the Marriott School of Management (mainly taken by students in information systems, marketing, business), but I think this might be the first time any university has offered a course specifically about its application in online education (although I doubt it will be long before others catch on). [If you want to get an idea for what is happening, here is a link to our class space.]
What are my expectations from the course? That everyone in the seminar:
(1) has hands on experience with the application of it with a real case,
(2) has an opportunity to contribute to an academic publication on the topic, and
(3) that our experience together produces knowledge and insights that are greater than any one of us could have come up with on our own.
My experience so far indicates that using web analytics to make and test recommendations for a simple online shopping experience (with metrics like CPC and ROI, measured in $) is a little more straight forward than deciding what set of data to monitor in order to improve online learning environments. But lets face it, with most e-learning there is a lot of room for improvement! -It is pretty easy to see how there must be some beneficial ways that web analytics can help to monitor activity and create insights for recommending and testing data-driven improvements. The initial question of importance is to determine the right KPIs (Key Performance Indicators).
Each of the participants in the seminar have already generated some ideas that I think will prove fruitful (click on their name to see their first brain storm of ideas):
Mary McEwen or here too
A combination of a summary of the class’s ideas along with my own thoughts over the last couple years of thinking about it will be forthcoming…
Questions for now:
For anyone who might be reading this blog entry –
– If you were designing and/or teaching online classes, for example, (as opposed to teaching one face-to-face and being able to see student’s reaction and adjust the experience in the moment) – what kind of things do you think would want to look at to help you make adjustments for the needs and experience of those you were teaching?
– Ideally what kind of things do you think you would want to be able to measure and monitor in order to give you insights into what is going well and/or what could be improved – (and for whom)?
I wonder if anything has more impact on our future than the questions we ask?
First, if we take it on more of a micro-level, imagine going into any random meeting. You will see things differently and have a different experience if you are asking “How can I get out of this meeting as quickly as possible?” vs. “What meaningful things can I learn and/or contribute during this time?” vs “How can I make sure I don’t embarrass myself in this meeting like I did last time?”
The questions we ask reveal some about the assumptions we take into the situation, and also have an impact on the consequent experience we have.
As another simple example, when meeting a person imagine asking: “What does he/she think of me?” vs. “What is his/her life like?” vs. “How can I make this person’s life a little better?” vs “Why am I even talking to this person?”
Depending on which question(s) you are asking (consciously or subconsciously) you will most likely have a different perspective, experience, and outcome.
As I was conducting a review the last 10 years of research on papers presented at the bi-annual CATaC conference (Cultural Attitudes towards Technology and Communication), I was again impressed by the questions we ask in a research context. They are all laden with assumptions (usually unstated) and have an impact on how the research is conducted – including what end up being the findings and recommended future research.
For this paper we looked at:
• Who is asking the questions? (where are they from, what discipline do they represent, who do they work with)
• What questions are they asking?
• How do they go about finding answers to their questions? (what literature do they cite, what methods do they use, what population do they sample, etc.)
• What answers do they find?
• What suggestions do they have for future research?
Additionally, I kept asking myself, what assumptions might they be making in the questions they address?
Even working with great colleagues like Javier and Brooke, it was a ton of work (reading at least some sections of all 199 papers) – but perhaps one of the best things I have done professionally or personally. I now have a better idea for what has already been done in this field, what gaps there are, and what lines of inquiry have been more fruitful than others. On another level, I am more conscious of the assumptions behind the questions I ask and the potential impact they might have. I wonder, out of all the possible options, are these really the most valuable questions?
• In your personal and/or professional life, have you ever had an experience where you noticed that when you changed the questions you were asking it altered the way you saw the situation?
• Do you ever stop to examine the assumptions you are making which led to the questions you are asking?
• Of all the questions you could ask, why did you pick the ones you are asking? Do you think they are the most important or valuable ones you could be asking or is it for some other reason?
As a side line of thought:
• Do you think we ask ourselves enough questions? Why as we age do we seem to lose some of the curiosity of children and ask less questions?
• If not all questions are created equal, how can I lead myself to asking better and better questions?
Just returning from Denmark (land of some of my ancestors), where I presented a paper at the Aarhus School of Business – “Knowledge 360” conference.
Perhaps the best thing about presenting my paper “Tools and Techniques for Online Cross-Cultural Knowledge Communication” – was that people in the audience knew about research and resources regarding cross-cultural innovation that I was not yet aware of. And it is always good to make connections with people who are doing interesting things which promise some potential of future collaboration.
One of the strangest things is that one of the most prolific faculty at the business school there, Connie Kampf, used to be the friendly girl serving me and my friends Orange Julius when we were teenagers at the Eden Prairie Center shopping mall in Minnesota years and years ago! (It is easy to remember because it was located near the arcade where we could get free tokens for getting good grades on our school report cards.)
In the last couple years, since we have both received our doctorates, I randomly met her in Malta, again in Estonia, now in Denmark and will see her later next month in the south of France!
Just goes to show what a crazy, small world this is – and that you never really know the potential or future of any ordinary person you meet on the street!
*So don’t give up on me just yet, I might one day do something worthwhile. 🙂 (No promises – but I’m just saying it is a possibility.)
I just got my student reviews back from this last semester (see below), and so I thought it would be good to give you a follow-up report on the Web Analytics class (ISys 590R) I was asked to teach. I talked about my unique approach to teaching this semester in a previous blog entry.
In this entry, I wanted to report on how that strategy paid off, and what I would do differently next time.
In the class we:
- covered some of the basics in eBusiness, SEO, online marketing, and the role and value of web analytics in it all,
- dissected web analytics implementation methods and discussed the strengths and weaknesses in each,
- participated in hands-on consulting experiences and data analysis using Site Catalyst (through the Omniture Web Analytics Competition) and Google Analytics (through required personal blogs, and in our case study),
- received exposure and personalized feedback to the experts who are on the cutting-edge of the field (through reading and commenting on expert blogs, guest speakers, participation in the Omniture Summit, and participation in the OWAC competition).
My two goals for the class were:
- create a sharing atmosphere where each person considered themselves a learner and a teacher (through in and out of class hands-on, immediately applicable projects and assignments),
- get students thinking like experts through as much association with as many as possible.
As a result of this class:
- many in the class have received job offers (several with Omniture, and several others with companies they have worked with during the semester as a result of their new skills)
- several in the class who have their own e-businesses have seen an increase in their profitability (some with really cool stories I could tell you)
- overall relationship with some of the major experts in the field has been strengthened
- during the OWAC competition 4 of my students placed in the top 3 teams, wining over $6,000
- several students said it was one of their favorite classes at BYU
- on the last day of class, the students gave a standing ovation (of course they were already standing because they were leaving, and maybe just clapping because they were glad it was over! 😉
Here are some examples of constructive comments that the students in my class last semester made on the anonymous university class evaluation form (they liked the course and the teaching style, but wished for a bit more structure):
Clint is a great professor. He is engaging, inspiring, and just overall, a nice guy. He did a great job of bringing experts in the field into the classroom. He was extremely respectful! Thanks Clint!
Clint is a great teacher. I like his teaching style. We’ve already talked about this, but I think the class needs a little more structure.
This is was one of my favorite classes taken at BYU and the teacher was one of my top 3 favorite teachers I’ve had in my 6 years at BYU.
First Clint was the best prof. that I have ever had in any collage class to date and Clint has left a high standard for any teacher to follow. He made himself very available out of class and maintained great communication out of class via email, and in class. He bent over backwards to get industry leaders in the field to come in and speak with us. WOW Clint is a fantastic teacher and is very open to feedback, almost more welcoming to negative feedback than positive. This great teacher was also aided by the fact that this is a great course which BYU is lucky to have. It will soon be a class at all university’s with any business school of any reputation. I am an electrical engineering major and was happy to come to jump the fence to come to this class. I would love to take Web analytics part 2. I took this class for the subject not because it at all helps me toward by ee degree or strategy minor. That is how important it was to me. I don’t regret it one bit. Two thumbs way up, for the instructor and the class.
This is the best class I have ever taken at BYU. It really helped prepare me for my career. I loved the Web Analytics competition. The professor did a great job making class interesting and keeping everyone involved. I learned a ton.
Maybe it’s the nature of the class but it seems like there was no way to see if a students schedule would be conducive to this class. I understand that during the competition we were busy but then it seemed like after we were done the class was scatter brained. Not the teacher just the class. It seems like assignments were assigned when the teacher felt like they came up, in the sense that as we were talking about different things suddenly he would say, oh great, an assignment! Then we were required to do it. It would have been nice to have a little more structure.
I did love the class though. I felt like I learned a lot and I loved having the guest lecturers. Thanks.
Good class . . . could have used a bit more structure to it, but excellent instructor!
I really like the professor, but I think the class was a little too loosely structured. I enjoyed the fact that I didn’t have to worry about my grade and I could focus on learning. However, there wasn’t much of a syllabus/course schedule that helped me know what was going on. He did send emails, which helped, but they were only for the next class period or two.
They are much better comments than the first time I got student ratings (about 3 years ago) – so it is nice to see some improvement 🙂 ). As you can see, however, the loose structure of the Web 2.0 approach still left students feeling a little lost. So, in consideration of ways to improve –
- I think there can be a little more structure and guidance, without encroaching on the overall teaching approach (e.g. give a little more details of upcoming events and assignments, request specific things to be blogged about instead of leaving it so much up to students, require in the first couple weeks for students to post a comment or question on an expert blog, etc).
- The other thing that I think would help is to start the consulting experience we do in the class before the OWAC competition, allowing data to be collected during that time, and then analyzing it and giving data-driven recommendations after they have their experience in the competition. This might also give the class a feel of a little more consistency throughout the semester.
It was very fun to teach ISys 590R this semester. I thought it was both personally and professionally rewarding. I hope those students keep in touch – if they keep learning at the same pace it will not be very long before they are the recognized experts in the field.
I also have some more exciting ideas for how to improve even more the tie with experts through the Web 2.0 tools – but perhaps you will have to wait until next semester to find out what those are… 🙂
Since I heard Seth Godin (a “guru” in online marketing) speak yesterday morning, I have not been able to stop thinking about some of his key messages. I’ll explain why I keep thinking about them at the end of this entry. I’m also very interested in your comments – what do you think are the best ways to get a message to spread?
The Old Way to Spread a Message: The old model of marketing was to try and interrupt as many people as you can with impersonal messages (through TV advertisements, magazine ads, billboards, etc) – and if you spent $1 getting your word out by interrupting people and made $1.10 in return, then you could spend it interrupting more people. Most CEOs and marketing people think that this same approach applies on the Internet and with online communication. Although this same (and frequently annoying) approach might still meet some degree of success online (in buying sponsored key-words, sending emails, putting up banner-ads) – ultimately the old model will fail in this new medium when head-to-head with what actually works.
The reality is that there are so many channels of information sources now that people can often ignore a company, even when it is spending billions of dollars in trying to interrupt you. Unless it is directly relevant or at least mildly entertaining, then they do not have time and they do not care. You can keep polishing your message, but it is simply a little pin in a wicked-huge haystack!
The main point:
Create something worth talking about. If you do not have that step, the next step will not mean much at all. (You can not buy attention, not effectively, not widespread.)
Ideas that spread, win.
In the middle (the majority) people strive to be average (only we live in a world where everything is usually good enough and we don’t have much time so we usually just pick what is either cheaper or closer), but on the edges people wait in line.
Definition of remarkable = worth making a remark about. If people remark about it, then the idea spreads.
Be remarkable (if you do not do this, do not go to step 2) – tell a story to your “sneezers” (the early adopters and innovators) – they spread the word (do what used to be your job) – get permission (the privilege of delivering anticipated, personal, and relevant messages – the kind that if they don’t come then people complain about not getting them).
There are two ways to get married: 1. Go to a singles bar, and the first girl you meet ask her right away to marry you. If she says no, then go to the next girl and ask her. If she says no, then go to the next one until you can find one who says yes (i.e. impersonal widespread invitations). 2. Find a girl, date her and get to know her, when you see there is a match then ask her to marry you (i.e. building a meaningful, welcome relationship). Most of marketing takes the first approach. The better thing to do is to create products, services, messages that people actually care about, and want to talk about and have more of. And of course, web analytics is one tool (of many) that can help people determine who is on the site, what do they care about, and how to customize the experience more on a one-to-one basis.
To read more of the details of Seth’s talk, you can read Kirk’s or Rob’s blogs (as they describe more of his talk) – or look at Seth’s new book: Meatball Sundae
Personal Application: I started to think about an idea that my sister and I have been working on for a couple months now. Originally we were just thinking about it in terms of a really cool children’s book (which I think could be a bit hit). After Seth’s talk, I started to think of other ways to use the technology available to customize, enhance, and easily spread it in a way that would make it something worth talking about. Does anyone who has programming skills want to find out more and see if you want to help me develop the idea?
Questions: Do you agree with Seth that the Internet has changed our lives in the ways mentioned? What do you think are the best ways to get a message to spread?
I heard Lance Armstrong, the famous cyclist and cancer survivor, speak last night at the Omniture 2008 Summit. Through sharing his story of surviving cancer, his multiple Tour de France wins, and his hugely successful LiveStrong movement (already sold nearly 70 million of those yellow bands) – his main point was to encourage everyone in the audience to do more to bridge the gap in society between what we know and where we actually are.
Funniest part: when he described the doctor who was trying to explain how simple his cancer surgery would be. The doctor enlisted the metaphor of Halloween – and had Lance envision taking a pumpkin, cutting the top off, carving out everything that was inside, and then just putting the top back on. Lance had testicular cancer! So it is understandable when he said he has never seen Halloween quite the same since then, and prefers if his kids ask his wife to help with the pumpkin carving.
Main Summary: He invoked the notion of active citizenship – or all of us being more involved in our community. He said that we know we need to because we are falling short, in schools, hospitals, homes, – we need to somehow shrink the gap between what we know and where we are at. “That is the gap between what we know how to do vs what we actually do – and everything in the middle is a moral and ethical failure in America.”
Speaking of the 70 million who have bought the yellow LiveStrong wristbands, he said it is nice to have an army of people who believe in change and want to do something about it. He emphasized that it was not just with cancer, but with so many things. He encouraged everyone to find the issue that they were most concerned about and then do something (even if not with money, then with time).
“We need your time, your energy, and most importantly your passion.”
Personal Reflection and Question: I think one of my key “issues” is intercultural (and interfaith) communication, collaboration, and innovation. It fascinates me and I think there is so much good that can be done through it for everyone involved. I think, however, that is part of my larger issue/passion – which is finding anything that helps people to see and reach more of their potential.
What is one of your issues?
For whoever reads this, pause for a moment and post something, anything. I am really interested to know what it is that you care about?
Please post something, the first thing that comes to your mind – I am really very curious.
The use of web analytics is not just changing online business, but all of business. That is one of the key messages that Josh James, CEO of one of the fastest growing and most innovative companies, Omniture, shared this morning at the opening session of the 2008 Omniture Summit.
Six years ago there was 6 clients (or rather potential clients) in attendance at the “Omniture Summit” this year there will be nearly 2,000 in Salt Lake (3,000 worldwide), from over 750 companies (more than 1,000 globaly – Sydney, Paris, Copenhagen, London, Munich, Tokyo) and those in one room representing more than 30% of the online marketing spend in the world. For example, this morning alone I have already talked to people from CNN, the NFL, Gateway, Microsoft, Convergys, HP, and on and on.
The reason web analytics is doing so much to change business (online and off-line) is simple. Because of the way in which everything can be tracked online – making ideas more measurable than ever before – this helps instill a data-driven mindset into an organization. Instead of “shooting from the hip” and arguing simply based on opinion, you can channel all that creativity into testing situations and consequently have data-driven results and recommendations for any decision you make.
The Omniture Summit will continue over the next couple days, and students in my ISys 590R Web Analytics class will be blogging about what they learn from:
- the keynote speakers (including people like Peter Kim – Forrester Research, Seth Godin – Marketing Guru, and Lance Armstrong – famous Athlete and speaker),
- consulting best practice presentations (in industry verticals like eCommerce, Media, High Tech, Financial Services, etc..),
- new tools and improvements on existing tools (e.g SiteCatalyst v.14, Discover, SearchCenter, Behavioral Targeting, tracking Video, Flash, Ajax, etc.),
- and success case studies (e.g. Ford, MTV, National Geographic, Blockbuster, Dex Media, BackCountry.com, etc…).
I feel lucky to be teaching one of the only web analytics classes offered at any business school in any university (yet one more reason why BYU is great). To read more about the incredibly valuable things my very cool students are learning and blogging about, see our online class space: http://ebiz2.byu.edu/analytics/
I believe that public school systems and universities are going to change significantly in the next 20-30 years. My thinking on these issues has been influenced by people like Lave and Wenger (Communities of Practice), Lev Vygotsky (Zone of Proximal Development), and George Siemens (Connectivism) – to name a few. As one of many changes, technology and the “new media” provides opportunities for students to tie into hands-on working and learning experiences through networks with experts in their field of study (and in some cases, even quickly become one of the experts in the field). Because there is so much to learn, and much that changes so rapidly, one of the most important skills we can teach our students (and learn ourselves) is how to navigate through the immensity of available connections and resources. I am currently experimenting with as many ways as I can think to do this with the Web Analytics class I am teaching this semester.
On the ITForum this week, George Siemens is discussing his paper: Teaching and Learning in Knowledge Networks
In response to a post he made today regarding the practical application of his theory, I described what I am doing in my class to apply many of the things that he mentioned. I am copying an adapted version of my post here for any who are interested:
The class I am currently teaching this semester is on Web Analytics. As it is a somewhat new and evolving field, I believe that the student’s I teach can even have an impact in leading and shaping it in the future. So I am doing everything I can think of to get them invested into learning from and contributing to the leading experts and networks.
1. I require them to blog – something that gives me a good feel for what they are learning (or not learning) in and out of class – believing it is something that taps into the idea that you are learning the most when you have to teach others. Sometimes it helps me get a feel for the personal side of student’s too, which helps in building rapport and trust in the class. And it is something that they will take with them after the class is over. [Incidentally it also gives them a chance to analyze the analytics from the visitors to their own personal blog throughout the semester]
2. We aggregate their blog feeds onto a central class site we created using drupal: http://ebiz2.byu.edu/analytics/blog
3. We also aggregate the blog feeds from other experts in the field onto the same class space.
4. Many of the class activities deal with contributing to the class space – for example, the class was split into groups to study and create presentations on certain aspects of online marketing – to teach the class and post their lessons online (e.g. through BrainHoney). When it came time to present, we had three experts come to our class and act like a panel – giving feedback and additional suggestions to the topics they presented on. We called it “American Idol of Online Marketing” and it was a huge success. All the experts asked for copies of the presentations, so we could direct them to the class wiki to continue to join the conversation.
5. In addition to having a lot of guest speakers, live or virtual, and reading the thoughts of cutting edge experts through blogs and forums, I am putting together an assignment to create a wiki page called “The Who’s Who of Web Analytics” – where class members will have a chance to approach experts, interview them with several short questions, and build at least some kind of a “warm” contact while simultaneously contributing to the class knowledge of who the experts are, what got them there, and what their current thinking is like.
6. We have a hands-on competition available where student’s analyze web analytics data from a fairly large website and make recommendations to the site owner based on the web analytics data how to improve it. They first give their presentations to actual web analytics consultants who pick the top four to present to the actual owners. This is good for class, for the company, and for the consulting organization – who has ended up making offers on the spot to hire some of my class members in the previous semester that I have taught it.
7. I have just negotiated with one of the biggest and best hands-on conferences in webanalytics to allow all of my class members to attend for at least one day. (side note: although I do not know what it has to do with Web Analytics, I personally am excited that Lance Armstrong is one of the keynote presenters.)
In previous semesters I just aggregated student blogs, but this semester I wanted to make a bigger push to get them tied to the actual community of practice. So those are some of the examples I have found helpful in doing that. Some of it has been a little more time consuming to get rolling, but I think now it is starting to be a self-feeding mechanism in a way. I also found one of the most important things I did this semester was to re-set the student expectations on the very first day of class away from what they might typically expect in a class and towards a new type of learning environment.
Of course, I am just learning in all of this myself – muddling through, making mistakes and hopefully learning from them as I go. But I also think it has been a lot of fun so far.
Thanks for your contributions to my thought process as I have tried these new ideas.
Any further suggestions or thoughts about my particular approach?
If so, they are always welcome.
I presented yesterday at the Shanghai Jiao Tong University E-Learning Lab, and was impressed by what they are doing. Although it is not the only thing they do, I quite simply have not seen anyone as advanced in mobile learning functionality – and I think it is because they have a nice combination of (#1) idea generation (hosting conferences and visiting with other scholars), (#2) technical expertise in developing the ideas (with the base of dozens of computer science students working right there), and (#3) the resources to make it all happen (including funding from Intel, China Telecom, BT, etc…).
Prof Shen showed me how students can view a live class room experience from their cell phone, toggling back between views of the teacher, the presentation material (if any), the live class, and at the same time send an SMS to the teacher in real time. I toured one of the “Smart Natural Classrooms” that make this possible. Immediately after each class session, they are archived and available for retrieval at any time, with students even being able to make calls to a call center for additional support as needed. The messages sent by email or SMS are first scanned with a natural language processor that extracts the meaning of the question and sends an automated response based on similar answers to previous questions. If that answer is not adequate, then the student can contact the teacher. The main idea behind it all is to have the class-room experience that people are already familiar with, but simply to extend the audience through mobile learning functionality.
The main current limitations seem to be that only one type of phone is currently compatible with the system (at least it is a Nokia :), but they are already working on extending the options of phones that will work. On the other hand, these lectures are a part of is a premium degree-granting program, so students are willing to pay more to be a part of the university, etc. The desire in China to have a degree from a top university probably can not be overstated. Also, my personal (admittedly somewhat biased) opinion is that they should be using technology not simply to mimic and extend existing class-room experiences, but also leverage the opportunity for student’s to engage in more web 2.0 kinds of learning and teaching experiences. Not just consuming content, but also creating, synthesizing, sharing…
My presentation to the lab today was in two parts: (1) Web 2.0 Paradigms & Platforms for Harnessing Collective Intelligence (with case: Agillix BrainHoney), and (2) Web Analytics and Decision Automation in E-Commerce and E-Learning Contexts (with case: Omniture/ Touch Clarity).
Afterwards – Dr. Minjuan Wang and I toured a part of old town Shanghai – enjoying ice cream bars – the treat in the center being sweet green peas (although Minjuan’s was sugar free, of course).
I just participated in the e-Learning DevCon 2007 conference over the last three days, and here were some of the topics and practical skills that I learned more about:
- How to create effective simulations for learning
- What Web 2.0 things do and do not work for corporate “knowledge management” purposes
- How to use video iPods for interactive learning (very interesting – never seen before how interactive an iPod can be)
- “Google gadgets” for learning
- How to create video blogs (vlogs) using a software called Vlog It! (which I hope to do soon!)
- How to create custom search engines with Google or EBSCO
- Effective ways to utilize and promote “drivetime podcasts”
- More about effective interactive design
- Overview of all the open source learning software and resources
- Using cookies to track learning data in Flash
- The power of video ethnographies in e-learning
- Must read book lists
- And a bunch more…
To be honest, the conference started a little slow for me, but by the end I was very glad that I went.
Machines really are able to act more intelligently due recent discoveries in mathematics!
Paul Phillips’ presentation was fascinating to me as I could instantly see so many connections in how the application of decision automation can significantly improve multiple areas of e-business and online learning.
Much of the history of communication has been finding ways to do the same things we have done in the past – but more efficiently. For example, the focus is usually on how to get information to humans more quickly and efficiently. But the problem is humans only have a certain “bandwidth” – and we now face information overload (e.g. Paul estimated that 70% of his work day is spent responding to emails).
Paul pointed out how certain domains allow for the computer to receive information/data and make decisions more effectively and quickly than if a human needed to receive, interpret, and execute action on the data. Obviously certain decisions won’t and should never be made by a machine, but with certain things it just makes sense to let the computer do it faster and more effectively than any human could.
The domain he picked for this lecture is online advertising, although it could apply in a number of areas. It is true that if you go into a store or restaurant multiple times, most likely the employees there will remember you and customize their service to your needs. Most web sites do not…yet.
The mathematical algorithm (much of which has been invented in the last 7 years) which makes this possible allows the computer to take into account huge amounts of data (6 “buckets” of variables) which are used to predict and display the most likely messages or creatives to lead to conversion (in the case of online advertising) while at the same time constantly testing (and making alterations based on the) risk that something else might be better. In other words, the computer is constantly using the what it predicts will be the best thing in the immediate situation (specific person at a specific time in a specific place with a specific known history) while simultaneously measuring and monitoring the risk that it might be wrong and something else might be better.
The results are measurable too. There have been astounding uplifts in conversion rates of certain sites that have already employed this method (sometimes over 100% uplift) when compared to random selection of messages/creatives.
I’m currently in the process of studying the mathematics and methods behind decision automation and will continue to post what I am learning…
I had a friend ask me how to use web analytics to improve the website for his business. Here are some very rough thoughts I had this morning. Keep in mind that certain web analytics vendors provide more on some of these features than others.
I am posting these ideas to my blog so that I can get feedback from other web-analytics users or professionals on their thoughts. Again these are rough thoughts…some might be more obvious than others.
Ten of the top ways web analytics can improve your website’s business (in making data-driven decisions):
- Significantly increase knowledge of who visits the site and improve understanding of when they come and what they do while they are there (time spent on page, pathing traveled, etc.)
- Distinguish between first time and returning customer behavior
- Utilize exit page and fall-out reports to identify what parts of your conversion process are difficult or uninviting
- Increase knowledge of where visitors are coming from (what search term, web-site, bookmarked, etc) – as an indication of what they are looking for and how to increase the relevance of your messaging
- Maximize available data on marketing ROI – conversion rates from different referral strategies (natural and paid search engine key-words, affiliate marketing, viral marketing, banner adds, email campaigns, etc..)
- Easily find broken links – increase usability
- Increase sophistication of segmentation and targeted messages – increase relevance
- Use A/B and Multivariate Testing to derive data on hypotheses of how to improve any aspect of your site
- When using Flash – gauge other usability issues through more nuanced data (e.g. how many times did users scroll over an item with their mouse before clicking it, how much of the video did they listen to, how long did they wait before realizing they needed to click on something, etc…)
- Leverage the power of the computer algorithms in taking into account wide variety of information about the user and predicting what is the best “creative” to serve (e.g. see Touch Clarity’s services – now acquired by Omniture)
Those are the first ten that I thought of.
What other ideas do anyone have from their own research/experiences?
We had Curtis Morley come to our Web Analytics class and share his expertise on action script for coding Flash. It is now time when we can see not only what people click on, but what they hover over and how many times, how long it takes them to click something, how long they watch it before clicking something else, and so much more. It lends more information toward what people are thinking, not only what they are doing.
He pointed out how the current examples posted on the Google help page work, but are flawed in a way in which you would not want to use them. He is working on a solution that promises to be much more user-friendly.
The whiz-kid Jimmy Z, from our Web Analytics class, created a plugin so that you can have shopping cart features inside of WordPress. Here are the steps, but you will need to get the plugin code before you do anything else. Once you have the code, here is what you do:
- Sign up for a Merchantec account (and a Google Checkout account).
- Upload the “JZ Merc – Products” plugin into the wp-content/plugins folder
- Go to the Plugin tab and activate the “JZ Merc – Products” plugin
- Create Shopping Cart Page in Write > Pages
- Put ‘view cart’ code from Merchentec into the code of the Shopping Cart page code
- On the Merchentec site, add specific items that you want to sell
- Create button code in Merchentec – using the URL from the Shopping Cart page.
- Go to Manage and click on the “JZ Merc – Products” plugin – add the ‘add to cart’ code
- Take the new code – put it in either a page or post.