Category Archives: Main Study

BSRLM abstract

I have to put together a 300 word abstract for BSRLM by this afternoon (well deadline is tomorrow but want to get it done today) and I have no idea what to write. I want it to be based on my thesis but the problem is that it is so difficult to cut down something in my thesis to 300 words. I am trying to determine what would be most relevant to BSRLM but can’t figure.

Anyway, I thought I would write something on the analytical framework I used for analysing how students performed on tasks. So, I’m going to use this space to write this. So, here goes.

This paper looks at the development and implementation of a framework for analysing students’ performance on tasks when using mathematical software. Through a literature review, three approaches and one affective factor were identified based on students’ understanding for analysing how students solved tasks and their effect on the students’ performance. These three approaches are 1) processing level approach (deep or surface learning), 2) self-explanation approach (quality of explanations), 3) explorations (extent of using software). The affective behaviour used was mathematics confidence. Using these approaches and affective factor a triangular framework based on the literature was developed which showed that all of these factors were related to performance and their inter-relationships. The literature indicated that only confidence was related to the processing levels but there were not any studies that showed that self-explanations were related to confidence, processing levels or exploration using software. 

Observing 38 students solving conceptual and procedural tasks using software whilst thinking-aloud, the research aimed to determine whether there was a relationship between these approaches and the factors using both qualitative and quatitative data. An updated framework was developed based on the results from this study. Some evidence was found that quality of self explanations was related to the processing levels.  Mathematics confidence was found to be related to software explorations and the quality of self explanations. Software explorations were also found to be related to the quality of self-explanations. The framework can be further used for testing theories on the relationship between performance and how students’ make sense of solving tasks.

Leave a comment

Filed under Main Study, Skills/ Development, Writing

Chapter 6 and Conclusion

Mood:  not sure
Now Playing: I’d be lying (Chris Cagle)

Well, I sent off my Chapter 6 (not the greatest but a chapter!) – and John sent me back some comments – he wanted more information on my conclusions – ok my conclusion was woefully thin – so really got to bump that up, but I think I’ll be ok once I re-read everything and fix it up. He thinks it is broadly sensible … hmm … I wonder if he thinks that is enough for a discussion or whether I have to make a discussion separately, I’m getting confused.

At the moment trying to write a conclusion chapter to figure out what I really found out and how does this become original research. 

Oh well, this is just a short post in trying to sort my thoughts out.

Leave a comment

Filed under Analytical Frameworks, Data Analysis, Main Study, PhD, Supervisors, Writing

I just may have a theoretical framework!

Mood: spacey spacey
Now Playing: Tapping of keys in an Open Plan Office

Well, I’m back from holiday and my one week bout with dengue fever (not pleasant!) – still feeling a bit lethargic from it (energy levels are really low). Anywho, just had a meeting with Doug and it has cleared up some stuff I was wrestling on. I was trying to figure out how to explain my qualitative data and I was heading to nowhere land fast because I was already digging up work in the affordance literature and stuff to do with Krutetskii (with gifted students) – which come to think about it I can still use but more on why I can on that later.

Well, Doug said I should forget about trying to look for a new theoretical framework and just go back to something that I was familiar with and suggested using the deep and surface approach and once he said it just made complete sense – that was definitely something I could use in the explanations of what was going on.  And so, had to check and see what I could find on confidence and deep and surface approach because I think that is the key in analysing most of my data even those in the quantitative analysis. Thankfully, there is something. The RASI questionnaire has a scale for academic self-confidence and Duff (2004) have found that students’ academic self-confidence was related to a deep-approach – now this is good news because noted that students who had high maths confidence were using more of a deep-approach – well, not statistically speaking with respect to the ASI, but some of the high maths confidence students were using a more deep-approach.

Not quite certain how it might fit in with the boxes but Doug suggested that perhaps one will suspect with the open-box and glass-box, students with a deep approach may delve further in trying to understand what is going on, but also perhaps this is only limited to the procedural part, perhaps in the constructive parts students with the deep approach in all three boxes may approach it similarly or be hampered (??) – I have no idea what I’m saying now – I think I’m just bluffing 🙂 .

Leave a comment

Filed under Analytical Frameworks, Data Analysis, Learning and Study Styles, Main Study, Meetings, PhD, Supervisors

Probably my main findings!

Mood:  a-ok
Now Playing: This is the night (Clay Aiken)

I haven’t posted something in this blog for so long and I’ve been procrastinating doing it as well – just like writing my thesis I guess. Anyway, I think I’ve finished all my data analysis and currently writing up Chapter 5 (yes I know I’ve been doing that for 4 months now!) – but hopefully it will finally come to fruition.

I think there are some fantastic findings (according to the quantitative data analysis) – here are the summary of my findings as I told Doug yesterday during my supervision meeting:

  1. Students who used the black-box software were more likely to score higher than those students using the glass-box and the open-box. Although students using the black-box seemed to perform less than students using the glass-box and open-box in the interpretive tasks (not certain if this is significant as yet – since I have to do a contrast test).
  2. Further, students who had a high maths’ confidence performed better than students with low maths confidence in the interpretive task, but both groups performed equally well (or equally bad!) in the constructive task.
  3. Further, it seems that when it came to students exploring with the software (i.e. testing numbers etc.), students who were using the black-box and had a high maths’ confidence were more likely to explore. However, interestingly – students using the open-box and the glass-box software and who had a high maths confidence hardly did any explorations in comparison to the students with the low confidence. In fact, the students with the low maths’ confidence who used the glass-box and open-box did more explorations than students’ with low maths’ confidence using the black-box (but got to check and see if these explorations amount are significantly different).
  4. Students who started with the abstract problem (Problem 3) made less explanations (both maths and real-life) than students who started with the application problems (Problems 1 and 2). Possibly here is something about moving from concrete to abstract or vice versa.

I think these findings are looking good and definitely something worthy of a PhD thesis (well I feel so 😀 ) – and I should continue to do so if I expect to keep my optimism in making sure my PhD is worthy – my only one worry now is finishing the writing up by September!

1 Comment

Filed under Data Analysis, Main Study, Meetings, PhD, Supervisors

The world of repeated measures and covariates!

Mood:  a-ok
Now Playing: Porpoise song (The Monkees)

Well, I’m back from holiday about 2 weeks now, and decided I should really post something in this blog.

Problem: Between Subject * Within Subject Interaction After Adding Covariate

I’ll just give an update on where I am at this point in time. Well, previously  I was using maths confidence and order  as covariates in the repeated measures in the box*problem*task for total scores (for which I was getting a significant difference for the box*task interaction). John indicated to me that I really shouldn’t be using order as a covariate since it was a nominal variable. I had only decide to use maths confidence and order as covariates since there appeared to be a significant difference between maths confidence depending on the order that students answered the questions. Further, he indicated that there should not be a difference in the box*task interaction as the covariate should only affect the between subject and not a within subject, and hence should not affect the box*task interaction.

Well, first of all, I wanted to test to see whether if I repeated the repeated measures with only the covariate of maths confidence to see what I will get, I tested it out and I still got a box*task interaction. But I was worried because according to John, this shouldn’t be happening. So, I looked up what this meant.

Possible Reason: Assumption of Homogeneity of Slopes Violated

I came across an editorial letter by Gilmore (2007) who was saying that the ANCOVA done by Anstey et al (2006) on their work on cataract removal, they used ANCOVA’s inappropriately, since the ANCOVA doesn’t follow that of Winer (1971) and that they included the covariate in interactions terms and indicated that this a problem with the SPSS (Resolution no. 22133). So, I looked up this resolution on SPSS knowledgebase and to be fair to SPSS and to Anstey et al (2006), the way they have written it doesn’t show in anyway that the ANCOVA or MANCOVA done is wrong but if the Winer (1971) ANCOVA has to be done, one should do it two parts. First do the ANCOVA first with the covariate to get the F-ratio for the between subject, and then remove the covariate and then run the ANOVA to get the F-ratio for the within subjects and the interactions.

That got me worried, because if I did that there would in no way be an interaction between box*task (task was a within subject) and I had already done this without the maths confidence as a covariate (and I was fairly convinced that maths confidence was influencing this behaviour). Well, I looked up back to see the reply by Anstey et al (2007) to Gilmore (2007), and they said that using the SPSS calculations was perfectly fine and it actually was a more cautious one as it allowed one to see if there were any violation of the homogeneity of the slopes assumption (an assumption that is necessary in ANCOVAs). A violation of this assumption occurs when there is a significant interaction between the within variables and the covariate. So, I checked back my repeated measures MANCOVA and sure enough there was an interaction between the within subjects (problem and task) and the covariate variable. That got me scared since I had no idea on how to deal with it. Well, looking through the internet I found a paper by Delaney and Maxwell (1983) which suggested a way of dealing with heterogeneity of regression based on a method by Rogosa (1980) which looked a bit complicated – it involved picking points and doing some long calculations with them, so decided to ignore that and try something else (I had to go search the internet for this paper again – because forgot to save it – but after a long search found it eventually!).

Solution: Recode Covariate

Anyway, so I looked up at alternatives way to deal with covariate variables, and one website had suggested that you can split the variables into groups. So, I decided to recode the maths confidence variable into two groups. Now I had done this previously arbitrarily (coded 1-5 as low and 6-10 as high) – but wasn’t certain if that was the correct approach. So, decided to do a frequency distribution of the maths confidence, and noted there were two high points one at 5 and one at 7, so decided it was a sort of a bimodal distribution. Further when I looked at the mean and the median, I noted the values were just over 6. So, decided to recode the maths confidence from values 1-6 as low and 7-10 as high. Whilst, just searching for the Delaney and Maxwell paper, I found a paper by Owen and Froman (2005) who were not keen on people ‘carving up’ their continuous variables. They did indicate one the most legitimate reason for this is in a ANCOVA where the assumption of homogeneity of slopes has been violated, but they still recommended trying to use the continuous variables perhaps using multiple regression instead, although they did recognise that this was a whole lot more complicated when dealing with repeated measures. They were also against using the single item for measuring something (as I did for maths confidence), instead they think a series of questions with a mean score would be better – and whilst I agree with this, but given the constraints of time on the study i.e. 2 hrs, I don’t think imposing a 10 item for measuring maths confidence would have been that much more useful.

Complication: Is the power enough?

Anyway, after recoding the maths confidence into these two groups I got 19 students each for the low and high confidence group. Whilst that sounded good, one of my worries were how were the confidence distributed across the boxes, and unfortunately for the black-box low confidence there was only 4 students and for the glass-box high confidence there was also 4 students, the remaining sub-groups had a distribution of 9, 9, 6, 6. So, those seemed safish enough, but wasn’t certain if this sample size will have sufficient power to detect a difference if there was. This looked completely complicated now, because now only did I have to do calculate power for a repeated measures design but now in which there was unequal sample sizes!

Well, I went back to Lenth’s power calculation java applet and tried to see if I could do it there, it if was possible, I surely couldn’t figure out what to do! So, gave up on trying that. I did a search on the internet on whether there were programmes that calculated power for repeated measures design, and I came up with this programme called PASS, which I downloaded for a 7 day free trial period and ran all of my variations with sample size, and thankfully, according to that programme there was a sufficient power for all my interactions (>0.9), and I tested also what should have been my minimum sample size for each subgroup, and according to PASS, I could have done alright with power when having 2 persons in each sub-group (>0.8). I don’t think I was able to figure out whether PASS could tell me whether there would be sufficient power to determine which group the difference is coming from when it came to the interactions, but that doesn’t make a difference to me as SPSS doesn’t do interaction differences when doing a repeated measures.

Outcome

Anyway, now that I knew that there was sufficient power, I proceeded to use my recoded maths confidence as a between subject, and there the box*task interaction was found as previously! All good! Felt much better once all of this was sorted. I told John about my recoding, and whilst he doens’t quite like it, he thinks in this case, it is alright just to get around the violation of the assumption. I then did all of my other repeated measures MANOVA using maths confidence and box as my between subject variables, and did these for types of explanations as well as explorations, and got some interesting results, which I feel much happier about writing … well, once I get motivated about writing it 😀 .

3 Comments

Filed under Data Analysis, Main Study, PhD

Data analysis – why isn’t there any significance!?!

Mood: hug me hug me
Now Playing: Nothin’ but the Wheel (Patty Loveless)

I have been bogged down (or trapped) in doing my quantitative data analysis. I can’t find anything special or significant about anything really. So, that is a bit depressing.

When I check overall statistical differences at a higher level (multivariate analysis) – I can’t find anything of real interest (well when I say anything of real interest I mean with respect to the boxes). When I look at the mean scores in terms of explanations that is being elicited for the boxes there seems to be a difference but when I run an ANOVA there isn’t any (the glass-box seems to have low maths explanations and high real explanations).

I do think I just need to double-check my consistency in scoring explanations  for the answers given – because when I was doing it I didn’t have a rubric as I did when I was scoring the marks. Since I scored the explanations on about 3 separate occasions with some weeks apart I just want to make sure there is some consistency. I think I need to develop a rubric and determine what I mean by real-life explanations and what I mean by mathematical explanation and what can be scored or not because I remember when I was scoring I did change the scores for some students I did previously which implies to me I don’t have a clear idea what it should be. Probably I should do it now since I’m in a writing frame of mind. Although the truth is in scoring these two explanations I do get confused for example one of the students (Participant 2) said for Problem 1C:

More soldiers would be sold as this means that the price of trains would have increased and the company would now have to make more soldiers

I mean this implies to me they’re explaining this from real-life since their explanation was definitely not based on any maths they previously calculated but using the word ‘increase‘ implies that is also mathematically motivated in some ways. I guess I’ll be turning to discourse analysis – I need to make clear how to categorize these or I would end up in a lot of trouble. In this particular case, I had only classify it as a real-life explanation.

Hmm … wonder what the criteria would be for real-life explanations and mathematical explanations – I know sometimes an answer can be scored as both but just want to make sure I have it clear in my mind. Ok – let me make a rubric now.

Mathematical Explanations

For mathematical explanations I think the following should hold true, the student has to have:

  • Written a mathematical equation e.g. 2x + 3y = 20
  • Used inequalities or equalities to help the explanation e.g. x<=20; y =5; x is greater than y. That is they must say something like “Since x<=20 and y=5  then ….” – in other words an argument has to be ensued from writing it in this way
  • Calculated a value or some indication of calculating a value e.g. z will be negative; 40-20 = 20; x approaches infinity; no change to x and y
  • Cannot be when student’s rewrite numbers that have been previously calculated e.g. number of chairs produced = 20
  • Cannot be counted as a mathematical explanation when they explored except when it is written in the answer sheet to indicate they have explored and give an explanation as one of the above. 

Real-Life Explanations

Mathematical explanations were somewhat easy to define – let’s see if I could do a real-life explanations – it seems to me my idea of real-life explanations tend to be the obscure explanations but explanations that seem to draw from life-experiences. Ok, real-life explanations are when:

  •  Not based on any explicit maths such as numbers or equations but using rules of thumb or heuristics that is commonplace such as “increasing production would increase profit”
  • Drawing to conclusions without testing any maths but rather what they ‘feel’ might be true e.g. “the demand is too low to have any production”
  • Drawing from their own feelings or experience on how the manufacturing world works e.g. “this would not be economical”

Additional thoughts 

Problems can only be coded into maths or real-life explanations in the part where students were asked to elaborate or give and explanation not when answering the question where they are required to regurgitate numbers. I am not certain if all explanations should be coded into these two categories, I mean I think I could leave some explanations as not coded. But is that right?

I think I’m still a bit uncertain about my rubric – I am having a hard time trying to code the following (Participant 8, P2B):

It does not change what will be manufactured because the other variables or constraints have not been changed.”

I initially coded it as a mathematical explanation but doesn’t fit quite so nicely into my mathematical explanations rubric. However, the way it is written implies to me it fits into the real-life explanations (based on the rubric) – but what is so real-life about this explanation? I get the feeling it is more mathematically oriented. I have sust it out in my mind and I think this should be relegated to a real-life explanation – now that I think more of it it feels as if someone sort of realised their amount of resources did not change which is not quite mathematical but more real-life. But perhaps it could be both since “no change” is a mathematical way of writing things that is the difference is equal to zero – I should add that onto the list.

I guess in this case, this explanation can be both a real-life and mathematical? My mind keeps changing – but the participant says the “variables or constraints have not been changed” is that the same as saying no change? Why am I quibbling about this – I hate this part of qualitative data. I don’t think this sentence means ‘no change’ as I’ve just written in the list. I’ve spun it through my head again and I’m going with this being a real-life explanation as there is no explicit reference to a constraint or variable and “have not been changed“; implies to mean a look at all resources and knowing that they have not changed and this perhaps would influence the results and thus aligns with the second point on the real-life explanations rubric.

Leave a comment

Filed under Data Analysis, Main Study, PhD

Data Collection – More Worries!

Mood: hug me hug me
Now Playing: Audio from my data collection

I’ve been trying to transcribe my data and also synchronize my videos. I find it easiest to synchronize my videos with audio using Dubit and inputting the wav file. Although got to use Audacity to convert the mp3 into a wav file – and it makes it huge! Dubit only uses wav files unfortunately.

Anyway, was trying to synchronize some audio and video this morning, when to my utmost horror I found that for two persons (one in GB5 and other in OB7), the audio was missing. Well, the audio wasn’t actually missing but rather there was a flat line for the participant and my audio was recorded. The beginning and ending of the participant’s audio was recorded it was just the middle. You know, the important bits where students were making the self-explanations! I think perhaps because the internet was so slow and using Skype when I clicked on screen capture with Camtasia, the audio stopped recording (that’s my conclusion at least!). 

Anyway, not sure what I should do now 😦 … I was just wondering if I could get around using just the written data and not the audio data for these two. I’m wondering if it will make a difference or whether I should just collect for two more people (but I don’t want to!!! – I’m fed up of data collection and I’m not certain Amalia would want to do anymore).

Transcribing itself is going pretty slow and that’s get me a bit frustrated especially with the need to write my ICME paper by the 20th April rather than the 1st June as was first advertised – it means I got to get it going pretty quickly!

Leave a comment

Filed under Data Collection, Main Study, PhD, Writing