Rock and Roll!

It’s the middle of December. Students have just finished their exams and between parties, I’m tying up some loose ends before I begin trying to clean up my office. I always clean my office during the Christmas break. My style is to create piles of files during the term instead of filing things away properly. As a result, by the end of the term I have massive piles of stuff that either need pitching or filing (most often pitching). Holidays are a good time to do this. I sort things into neat piles by topic throughout my office and the neighbouring offices and then pitch or file as required. As I sort through the stuff that I’ve saved I’m usually reminded why I saved it in the first place. All this is done to the tune of rock and roll coming from my iPod! This year I’ll use the clean-up process to up-date some of my blog postings. Some of these relate to my job in Student Affairs – others relate to my radio program or CDs that I’ve reviewed.

As I mentioned in my very first posting, I host a radio program on 93.3 CFMU . Mine is a “soul music” programme and I try to have a theme each week. Last week, for example, I did a special on Ike Turner, who passed away on December 12^th. Unfortunately Ike is best remembered for drug abuse and for abusing his wife Tina Turner. He is however credited for what some consider to be the first Rock & Roll song, “Rocket 88” in 1951. Ike was a great guitarist in his own right and won a Grammy award as recently as February 2007 for best traditional blues recording.

On Monday the 24^th my program will be all Christmas music. This is actually a pretty easy program to put together since most soul singers came up through the church and gospel choir route. Once they made a name for themselves they started to record a Christmas song once a year or so. Even James Brown has one  - “Soulful Christmas”. Of course in the song James brags about his music and thanks everyone for attending his shows. Christmas Day will mark the first anniversary of the death of James Brown. I will have a two hour special on CFMU that day. I’m quite proud of that program as I invested a lot of time into the research and found some new CDs to use. It traces Brown’s  life  from his time in reform school through his gospel days with The Flames, his soul contributions (he’s the God Father of Soul) up to his creation of funk. One thing I learned is that the drum beat from Brown’s “Funky Drummer” has been sampled more than 100 times on modern hip hop tunes.

A re-constituted Led Zeppelin played a concert at London’s O₂ arena earlier this month. Apparently the concert was great and people travelled miles to see it and paid a ridiculous amount of money to see it. The concert brought back many memories for me. I saw Led Zeppelin at the Aud in Kitchener in the Fall of 1969. In an article in the Toronto Star, Greg Quill remembered that 1969 tour: “But other fans are recalling three Led Zeppelin shows here 38 years ago that set this town on its ear – on Feb. 2 and Aug. 18, 1969, at the 1,200-capacity Rock Pile (formerly the Masonic Temple, now MTV’s headquarters) at Yonge St. and Davenport Ave., and a third at the 4,000-seat O’Keefe Centre Nov. 2.” The concert on Nov. 2^nd  was part of the tour that I saw on Nov. 4^th . It was awesome for a 20 year old hick from Brantford to see. Zep had just released their second album (Led Zepellin II on Oct. 22^nd ) and it was on sale at the event for $5. Looking back, it looks like I paid just $4 for my ticket in the “blues”. (Another side note to Greg Quill’s article is the reference to the “Rock Pile” in Toronto. The Rock Pile was the inspiration for the “Gas Works” in Mike Myers movie (Mike is a Toronto native) – “Wayne’s World”.

Another Toronto landmark, The Horseshoe recently turned 60. This event was also documented in the Toronto Star by Greg Quill. Again quoting from Quill’s article: “From its post-war origins as a honky-tonk hosting the likes of Willie Nelson, Waylon Jennings and Loretta Lynn, through its Canadiana years as Stompin’ Tom’s stomping ground in the late 1960s, to its punk glory days a decade later under renegade promoters Gary Topp and Gary Cormier, to its ascendancy as the birthplace of Canada’s new rock heroes (The Tragically Hip, The Watchmen, The Bourbon Tabernacle Choir, Moxy Früvous, The Barenaked Ladies) in the ’80s and ’90s, and since then as a showcase for an astonishing array of diverse emerging talent from across Canada and around the world, The Horseshoe has never really lost its edge.”

One of my favourite stories about the Horseshoe comes from this article in The Star by Ben Rayner: “ KENNY SPRACKMAN, co-owner: ‘Several years ago, The Band was in town to receive a special Juno Award. The `Shoe had The Band onstage that Saturday, of course less Robbie Robertson. They had not performed together for many years.
About 12:30, the back door opened and in walked Robbie with his friend Reg Bouvier. I also noticed he had a guitar with him! My blood pressure went through the roof with excitement. Was I going to see a Band reunion in my club? Robbie opened up the guitar, put it on and entered side stage. The look of surprise on every one of the members on stage brought tears to my eyes and a roar from the crowd only equal to the night the Stones hit our stage.
I got so drunk from the excitement – and those who know me know I do not drink – that I opened up the bar and ran around for the rest of the night giving free shooters to all who would partake and consequently getting the majority of patrons drunk with me. I have seen many special things at the `Shoe but to me this was as good as it gets.”

I am a huge fan of The Band. I first saw them as Levon and the Hawks in 1964 and would loved to have been at the Horseshoe that night that to see Robbie strap on his guitar for an impromptu reunion. I saw the Robertson-less Band at Canada’s Wonderland (with The Grateful Dead!) in the mid eighties and they simply weren’t the same.

Maclean’s Rankings

Maclean’s magazine has been ranking Canadian universities since 1992. Although Maclean’s made the first rankings in Canada, they appear to have copied the U.S. News and World Report who have been ranking U.S. universities since 1983. As I’ve mentioned in previous posts, there are several ways to “rank” or “rate” universities. The Globe and Mail’s University Report Card (URC) and the Canadian University Survey Consortium (CUSC) use student satisfaction surveys. That is, students are asked to rate the various services of their university and asked to rate their overall experience. NSSE attempts to measure “student engagement” by asking students how they spend their time. Maclean’s and the U.S. News and World Report use “Performance Indicators”. For the benefit of readers who are not familiar with the Maclean’s rankings, Maclean’s places universities in one of three categories which are meant to recognize the differences in types of institutions in terms the of level of research funding, the diversity of offerings, and the range of graduate and professional programs. The three categories are Primarily Undergraduate, Comprehensive and Medical Doctoral. Primarily Undergraduate institutions are largely focused on undergraduate education and have relatively few graduate programs and are usually small. Universities in the Comprehensive category have a large amount of research activity and a wide range of programs at the undergraduate and graduate levels, including professional degrees. They need not be small either. Medical Doctoral universities offer a broad range Ph.D. programs and research and all universities in this category have medical schools.

For the first fifteen years of the rankings a similar set of performance indicators (PIs) was used. The broad categories were Students, Classes, Faculty, Resources, Student Support, Library and Reputation. Within each broad category was several sub-categories: Students (Average Entering Grade( 11%), Proportion with a > 75% entering average (2%), Proportion who graduate (2%), proportion international students (0.5%), International graduate students (1%), Student Awards won (3%)); Classes (Class size in first and second year (7%), Class size in third and fourth year (7%), Classes taught by tenured faculty members (3%)); Faculty (Faculty with a PhD (3%), Awards per full-time faculty (3%), Social Sciences & Humanities grants (5.5%), Medical and science grants (5.5%), Resources (Operating budget 3.3%); Student Support (Scholarships and bursaries as a % of operating budget (4.3%), Student services as a % of operating budget (4.3%)); Library (Total holdings (1%), Holdings per student (3%), Acquisitions (4%), Expenses (4%); Reputation (Alumni Support 3%), Reputational survey (16%)). The numbers in brackets are the weightings for each sub-category and I have listed the sub-categories as of 2005 (at least as I have defined them).

For many universities the ranking exercise was a year round activity. The process began in the spring when thousands of reputational surveys are sent to university officials, high-school principals and guidance counsellors, heads of organizations, CEOs and corporate recruiters across the country, asking for their views on quality and innovation at Canadian universities. As the AVP (Student Affairs) at McMaster, I am asked to complete this survey. I have a fairly good knowledge of the university scene in Canada having been involved for nearly 40 years. However, I can really only answer the survey honestly for a very few universities. I don’t know how the other recipients of the survey, the majority of them would have considerably less experience than me, can give very knowledgeable answers. For the first 16 years of the Maclean’s rankings the universities themselves provided all of the other information. Over time this lead to a number of issues that I will talk about shortly.

For the first six years of the survey McMaster ranked sixth overall in the Medical Doctoral category out of 15 schools. This overall ranking began to slip in 1998 as follows: 6^th, 6^th, 7^th, 8^th, 10^th, 11^th, 8^th, 11^th and finally 12^th in 2007. McMaster was badly affected by its ranking in class size (usually dead last and worth 15% of the total) and incoming grades. On the other hand our reputational ranking was usually in the top five and often in the top 3. It was the relatively high weighting of this category that kept us out of the basement. Over the years we were concerned that universities were not accurately reporting their data. We had reasons to doubt this and Maclean’s had no way to audit the input. There certainly was a way to “play the game”. There were also very legitimate ways to do better. When I took over as AVP we were ranked 15^th (of 15) in Student Support. The reason for this was that our university was under-reporting the amount of money that we were spending on this important area. One of my first tasks was to have my accountants (and those from around the university) determine all of the money that we were spending on student services using the broadest possible definition. Over the next few years our ranking in this sub-category rose from 15^th to 10^th, 9^th, 3^rd, 4^th, and finally 6^th in 2006.

Another concern of universities was the cost of preparing the data for Maclean’s. Frankly, they either asked for it in a format that we didn’t have it in or asked for data that we didn’t usually collect (like total student services funding). Finally a number of universities got fed up and pulled out of the rankings, refusing to submit their data. Initially there were four universities who withdrew (Toronto, Calgary, McMaster and Alberta) and ultimately 26 withdrew. Frankly, it was when the University of Toronto withdrew that the movement was acknowledged nationally. Toronto had been ranked first or second every year that the survey had been done and so when they pulled out, it couldn’t be attributed to “sour grapes”. Quoting from the letter from the President of the University of Toronto (and signed by 10 other Presidents): “To begin with, the Maclean’s rankings aggregate data from a range of variables related to the student body, class sizes, faculty, finances, library and reputation. It is inappropriate to aggregate information across a range of programs at a large and multidimensional research university into a single ranking number. Consider how such an approach might pervert one’s understanding of a general hospital that is ranked #1 in obstetrics and #10 in cancer care. Averaging these rankings would result in this hospital being ranked “#5 overall”. For the patient seeking care in one of these areas, such a measure would be useless at best and misleading at worst. This is, effectively, the method that Maclean’s applies to Canadian universities by its calculation of “league tables” based on the arbitrary assignment of weights to variables which, by themselves, are of questionable validity. The variables selected by Maclean’s also fail to capture the breadth of experiences students say are important in their university education such as, for example, extra-curricular activities or the opportunity for rich and diverse interactions with peers and faculty outside the classroom.”

Despite losing contributions from about half of the universities in the country, Maclean’s persisted and published its rankings in 2006. In calculating the rankings for 2006, they used the data for the 21 universities who provided the data as usual. For those universities who refused to complete their questionnaire, Maclean’s obtained public information where possible for the most recently available year – which wasn’t always the same year the other 21 reported on. In cases where no public data was available, Maclean’s simply re-used the 2005 data reported. Statistics Canada data was used for the financial indicators in 2006. CARL data was used for the library. In summary, the rankings for 2006 were a total fabrication – even more prone to methodological problems than in the past, being a mishmash of publicly available data for different fiscal years and recycled and re-used numbers directly from the non-participating universities.

I guess that Maclean’s saw the error in their ways because (according to this year’s issue): “This year, Maclean’s revised its methodology, and the rankings are now based entirely on publicly available data. Student and faculty numbers were obtained from Statistics Canada, as was data for all five financial indicators-operating budget, spending on student services, scholarships and bursaries, library expenses and acquisitions-as well as total research income. For the social sciences and humanities research grants indicator and the medical/science research grants indicator, data for fiscal year 2006-2007 was received directly from the three major federal granting agencies: the Social Sciences and Humanities Research Council of Canada, the Natural Sciences and Engineering Research Council of Canada, and the Canadian Institutes of Health Research. The Canadian Association of Research Libraries and its regional counterparts provided figures used for the library holdings indicators. All financial and library figures are for the fiscal year 2005-2006; student and faculty numbers are for 2004-2005. Exclusively in the magazine, you will also find display tables of additional data, such as entering grade averages and graduation rates-information that, surprisingly, not all universities are willing to make public. Maclean’s obtained the data in this section directly from universities, from university websites-whenever such data was available and comparable-as well as from Common University Data Ontario (CUDO), an initiative of the Council of Ontario Universities, and the British Columbia Higher Education Accountability Dataset (BC HEADset)”.

Low and behold, McMaster’s overall ranking jumped from 12^th in 2006 to 6^th in 2007! How did this happen? It wasn’t simply a matter of using publically available data. Firstly, Maclean’s dropped 10 indicators, many of which ranked based on minute differences in things like average entering grade. Many of these ten indicators were ones on which McMaster hadn’t done well on in the recent past, such as average entering grade and class size. These were indicators where some institutional “tinkering” probably occurred. For the remaining 12 indicators, the data are now derived from publicly available data or from the Reputation Survey conducted by Maclean’s. (Another problem with Maclean’s methodology was the “first past the post” mentality with would rank one university higher than another based on incredibly small differences in absolute scores).

Secondly, Maclean’s has added two new indicators. The first is student/faculty ratio, with data derived from Statistics Canada. They say that this is ratio of all FTE students (grad and undergrad) divided by full-time faculty. McMaster is middle-of-the-pack on this indicator. Maclean’s does not indicate how the FTE student is calculated. Division by full-time faculty is still imperfect as well, because of the inconsistencies involved in the reporting of clinical faculty. The second new indicator is one which measures TOTAL research, one for which McMaster has argued for. Thirdly, Maclean’s has changed the way in which they weight the various indicators. The weighting for the “students and classes” indicators has dropped from 40% to 20%. We also expected that there was much manipulation of the class size data. The weighting for the faculty section remains approximately the same, moving from 17% to 18%, but the “faculty with PhD” indicator is now gone. The new Resources grouping now a weight of 12% compared to 3.3% before. With a weight of 6%, the new research indicator definitely helped McMaster’s position as did the increase in Student Services from 4.3% to 6.5%. The weight for total student support is up from 8.6% to 13%, and as the data is derived from financial reporting, there is no more room for manipulation. The library indicators are also weighted marginally higher at 15%, up from 12%. Lastly, the reputation ranking is now worth 22%, up from 19% in the past. Also, the alumni support indicator is gone. This change in weighting was magnified by McMaster’s improvement on the reputation survey overall, and in 3 out of the 4 reputation indicators.

Those of us at the university who have been involved in surveys do feel some vindication in the increase in our rankings given that they are now based on publically available information. We are still of the same opinion of Dr. Naylor though and that is, that the Maclean’s indicators do not in fact identify the best universities. Universities have different missions and goals and are each strong in their own ways. Maclean’s ratings do have a wide readership though and are paid attention to by the public and universities alike. Some of their indicators are related to quality but students and parents interested in particular universities would do well to find out all they can about them and make sure to visit!