Two professors I had as a graduate student publishes a paper that helped me see how assumptions are embedded in any methodologies and/or scientific paradigms. Yanchar and Williams (2006) argue that the “metaphysics” - or assumptions underlying our science, methods, and theories - are too often held uncritically, because they are unconscious, and are thus transferred to others more easily than what is directly argued. In his discussion of the “metaphysical” assumptions underlying science, Burtt (1954) said:

“If you cannot avoid metaphysics, what kind of metaphysics are you likely to cherish when you sturdily suppose yourself to be free from the abomination? Of course it goes without saying that in this case your metaphysics will be held uncritically because it is unconscious; moreover, it will be passed on to others far more readily than your other notions inasmuch as it will be propagated by insinuation rather than by direct argument.” (p. 229)

Yanchar and Williams (2006) propose that although we cannot escape the theoretical assumptions we make, we can at least take efforts to make them more explicit and coherent.

Now exactly what that means - I am still struggling to understand myself? And how each discipline chooses to handle this issue seems to be a little different. I know those are complex ideas - but any other thoughts?

1) In case of my thesis, it was a bit hard to explain methodology, since it is paper-based work. Each paper had its own methodology, since they sometimes were devoted to subjects of a VERY different nature.

2) In case of educational science research, I understand what “methodology” is. Since you’re dealing with people, there is no 100% method to be sure about the results. There are only methods of different degree of reliability. So you can explicitly state that you’ve used questionnaires, formal numeric results or even lie detectors — this information might be extremely important to the people who will either believe or not believe into your results.

3) At least, in our tradition (in Russia), in natural sciences people are encouraged to avoid metaphysics. It is considered as something not scientific For example: “our experiments show that molecules behave like balls”. This is a wrong approach. You should say: “if we treat molecules as balls, then we can use technique X and Y to calculate Z, and the analytic results match the experimental results”. So you explicitly state that you build a model that helps to get simpler and reliable calculations, but you avoid making conclusions on how the real world is actually operating.

4) And I have no idea what is methodology in maths and algorithms analysis. To prove the compexity of the algorithm I either use maths (complexity analysis) or experiment by running the program watching the clock. There are definitely some initlal assumptions on the input data, used hardware, etc. But I wouldn’t call them “methodology”…

The only difference between the softer and harder sciences is that the softer ones are forced to consider these factors because their fields are too complex to be addressed by purely experimental approaches or reasoned proofs.

For me methodology is the approach how you’re trying to solve the problem and what set of tools and techniques you’re using while doing it. It is seems to be true that this “toolbox” is not documented in most of maths/physics and more mathematical studies in CS due to the fact that it is somehow obvious. When you read e.g. a mathematical proof, you will know how it has been done. E.g. if it is induction, it is clear that the author has used induction to prove it.

Maths is perhaps the most “precise” of these sciences. Physics is wide area and I do believe they sometimes really have to explain how the research has been done and how they have get to the results. CS is perhaps the most problematic of all these, since there is so many type sof CS. Just see what the dept. of CS at Joensuu is doing. Thus, I believe some effort should be put on documenting the “methodologY” what ever it is. For example there are several ways to study algorithms; you can prove it mathematically or you can test it with datasets. Thus, two approaches, two methodologies, a need to explain why and how. Again, this might be obvious while reading it, but still I think it be worth to document it explicitly.

However, Thomas Kuhn in his talk of “Normal Science” and “Scientific Revolutions” possibly gives some indication of the reasoning. Kuhn would contend that in periods between changes in the underlying paradigm (revolutions), the methods are taken for granted. They fit within the particular operating paradigm of the scientific discipline. It is only when there is disputes over the underlying paradigm that debate on methodology becomes relevant.

In Maxim’s case, he is operating within the defined parameters of the field of algorithm development. I haven’t read the thesis although I think Marilyn did but I would imagine that Maxim applied the standard techniques for evaluating algorithms such as performance, effectiveness, … In that setting, I wouldn’t expect any discussion of methodology unless he believed that there was a better way to assess the suitability of his algorithms for the domain that he was operating in.

The difficulty in the social sciences and education is that for any particular question or hypothesis, there are a range of possible ways of attempting to prove the validity of the hypothesis. Maybe, it could be said that there is no claim to normal science within these disciplines. I see computer science education research as also lacking a normal science paradigm.

Regards, Errol