Apostle of permissiveness: A.S. Neill, founder of the progressive Summerhill School and best-selling author 

Halfway through my first year as a history teacher at an inner-city comprehensive in England, I am reeling from the volley of abuse and misbehaviour that makes up my daily grind. I can be sure that at some point in my day I will be aggressively confronted, blithely disobeyed, and probably sworn at. Restless nights are common, and nervousness ongoing. Still, talking to my friends from teacher training, I feel I'm having a comparatively easy ride. I have not yet been physically assaulted, and so far I have avoided the much-feared mid-lesson breakdown. 

At schools such as this, the deprived background of the children is routinely presented as a catch-all explanation for bad behaviour. The pupils' chaotic home lives, their lack of prospects, and an absence of aspiration in the local community are all popular excuses for the pandemonium that pervades inner-city schools. These factors undoubtedly have an effect, but such thinking lets schools like this off the hook. The endemic discipline problem within the state sector is in reality self-inflicted. At least half a century of "progressive" thinking on pupil behaviour has had disastrous consequences. In the competitive field of follies wrought by 1960s radicalism, there is a very good case for progressive education being the most socially destructive.

My school's discipline problem is depressingly normal. A survey last October for the Guardian Teacher Network — hardly a bastion of old-fashioned disciplinarians — found that 40 per cent of teachers complained of being bullied by pupils and, of those who considered quitting, 50 per cent named pupil behaviour as the reason. A 2010 National Union of Teachers (NUT) survey found that 92 per cent of teachers believed pupil behaviour had worsened over the course of their career, and 79 per cent claimed that they were unable to teach effectively because of poor behaviour. During the last school year, 44 teachers were hospitalised with severe injuries from pupil attacks at a five-year high. Perhaps most worryingly of all, a 2008 Policy Exchange report showed that the atrocious reputation of British schools for poor behaviour was the main factor in deterring new graduates from becoming teachers. 

Despite the recent arrival of an energetic new head, my school's results remain stubbornly unimpressive. It is strikingly obvious to me and many of my colleagues that the fundamental impediment to pupils learning is a lack of classroom discipline. However, when I suggested this to a member of senior management at a training session, he winced at the very word "discipline". "Right," he said swallowing uncomfortably, "behaviour for learning" — this being the trendy euphemism, modishly abbreviated to B4L, favoured by schools too right-on to use the D-word. How, I wondered to myself, did British education get to a state where discipline is a dirty word?

In an essay on education written in 1961, the political theorist Hannah Arendt foresaw the steady erosion of discipline in Western schools. She wrote: "The problem of education in the modern world lies in the fact that by its very nature it cannot forgo either authority or tradition, and yet must proceed in a world that is neither structured by authority nor held together by tradition." If this was a problem in 1961, it is a catastrophe in 2012.

Since Arendt wrote her essay, legions of progressive educators have denied the need for authority in schools. The permissive rhetoric of 1960s radicalism was particularly influential among teachers, and their ideological precepts were applied to classroom culture. The undisputed leader of this "progressive" movement in Britain was A.S. Neill, founder of the revolutionary Summerhill School. Neill documented his philosophy in his 1962 book Summerhill, a runaway success which sold more than two million copies. In it he claimed, "I believe that to impose anything by authority is wrong. The child should not do anything until he comes to the opinion — his own opinion — that it should be done."

After the 1960s, radical educationists who subscribed to this thinking began their long march through the institutions. The idea of child-led learning came to dominate our teacher-training colleges and classrooms. Such thinking claimed that teachers should never coerce pupils to learn against their will, but instead place them in a situation where they can learn for themselves. The favoured description of a teacher's job changed from "teaching" to "facilitating". The rhetoric of child-led education was, and still is, extremely seductive, but it has failed to deliver. It is premised upon a fatally misplaced assumption that pupils can be relied upon to know what is best for them. The practical consequence of this utopian thinking has been the consistent fall in standards of British state education.

Over the course of the 1970s and 1980s, British comprehensive schools gained their reputation for ill-discipline and low expectations. One of the most articulate critics of these developments was the poet, teacher and literary critic, Brian Cox. Born in Grimsby and raised on Milton and Methodism, Cox was a working-class intellectual of the old school. Together with such luminaries as Iris Murdoch, Kingsley Amis and Robert Conquest he edited the Black Papers — a series of strident attacks on the changes happening in British education. In his 1992 memoir The Great Betrayal, Cox wrote that "the abdication of authority by teachers has fundamentally damaged our society".  

By the late 1980s, British comprehensives had become synonymous with chaotic indiscipline. A survey carried out in 1987 by the Professional Association of Teachers found that 94 per cent of teachers believed indiscipline was becoming more commonplace, 86 per cent believed that classroom violence was increasing, 80 per cent had been subjected to verbal abuse, and 32 per cent had been physically attacked by a pupil. For many, the link between the crisis in British schools and the radical ideas which preceded it was unambiguous. As Cox wrote in 1992: "Today the breakdown of discipline in inner-city comprehensives is a direct result of the sicknesses which afflicted the schools in the 1960s."

Tory reforms of this period focused on applying free-market principles to the running of schools, but allowed the philosophy of schooling to remain largely in the hands of progressive educators. By the time New Labour came to power, the inheritors of 1960s radicalism had firmly embedded themselves in the institutions, and progressive ideas about education received a new lease of life. The later director of Demos and leading light in Labour policy circles, Tom Bentley, wrote Learning Beyond The Classroom in 1998. On the topic of discipline he claimed: "Expecting young people automatically to accept someone's authority because they are in a position of power is unrealistic, as well as unhealthy."

It was these progressive orthodoxies that suffused the PGCE (Postgraduate Certificate in Education) training I received to qualify as a teacher. While at university, I received next to no lessons on classroom management. When we challenged our tutors on this point, the answer was always the same: as long as you get the other things right (pupil motivation, interesting lessons, positive thinking), behaviour should fall into place. We were not alone in this worrying omission: according to a 2008 NUT survey the majority of teachers have never received training in behaviour management. Considering just how much pupil behaviour dominates the concerns of a new teacher, this finding is startling. 

The sole session I did receive on behaviour management consisted not of discussing practical methods, but instead pondering the "root cause" of bad behaviour. "What is more important," we were asked, "in explaining bad behaviour at schools: absent fathers, or children not eating a healthy breakfast?" Such sessions seemed more concerned with making armchair sociologists of us than effective classroom teachers. 

Thus prepared, I was sent to an inner-city comprehensive where 37 per cent of the pupils qualify for free school meals and 42 per cent of the pupils have special educational needs. What I did not anticipate, though, was that the ethos of the school would actively militate against ensuring good discipline. The senior leadership team openly states their dislike of "complicit" pupils. As a result, the head walks the corridors with all of the authority of a dinner lady. In the staff room we trade stories of the complete lack of respect pupils have for our leader: "I saw him knocked over by two year nines!"; "Bradley in year 11 told him to fuck off, and walked away blowing kisses!" At a staff training evening, the head offered a defence of his outlook. He drew the hysterical moral equivalence between a compliant student and an obedient German soldier in Hitler's army. For him, good behaviour is oppression, strictness akin to fascism. 

According to the doctrine of child-centred education, we teachers should prevent misbehaviour by honing our lessons and teaching style. We are endlessly told that a good teacher is tough on the causes of misbehaviour, not misbehaviour itself. In practice, this translates into placating pupils and never really pushing them to achieve. Exhaustive efforts are made in schools to introduce new "behaviour management solutions". Fast-paced lessons; interventions; CCTV surveillance; behaviour tracking; child therapists; more assessment; less assessment; motivational training; bribes; even bouncers: a constant merry-go-round of "cutting edge" methods trying in vain to compensate for the abdication of authority in schools.

The paradox which afflicts schools such as mine is that when teachers are relaxed on discipline, discipline becomes their overriding concern. In strict schools where rules are consistently enforced, pupils know the expectations for their behaviour and teachers can focus on teaching. In schools where discipline is relaxed, ensuring good behaviour becomes an all-consuming battle. In my car journey home with two other teachers, behaviour dominates our discussions. We rarely get round to sharing stories of actually teaching as we are so preoccupied with getting the pupils to sit down, stop talking, open their books, and pick up a pen. I can only dream about what I could achieve with my pupils if we were in a school where good behaviour was the norm, not the exception. Thankfully, it seems that a corner is being turned. The government is keen to address not just the structure but the culture of state schooling. In 2007, David Cameron attacked the educational orthodoxies which were a "hangover from the 1960s" and said discipline was a key ingredient for successful schools. The proposed reforms to the Ofsted inspection process are going to reflect this, with a quarter of school inspections being dedicated to school ethos and behaviour. 

However, the most encouraging developments are happening on the ground. By now, the achievements of the new head of Ofsted Sir Michael Wilshaw are legendary. He took over one of the worst schools in the country, Hackney Downs Comprehensive (renamed Mossbourne Academy), and imposed a  regime of homework, uniform codes, regulation haircuts and silence in the corridors. The result? An astonishing 84 per cent of pupils now get five good GCSEs including English and maths. Seven of his pupils went to Cambridge University this year, including one who became a single mother at 14. Similarly, a string of academies run by the educational charity ARK are achieving mind-boggling results, with firm discipline a key ingredient of their success. ARK also plays a part in Future Leaders, a headship training programme which is beginning to roll out a new generation of heads, dedicated not to progressive fantasies but pragmatic solutions. Observing these developments from the frontline, I feel genuinely buoyed. 

During the 1960s, there was no end to the promises made by progressive educators: harmonious schools, emotionally fulfilled pupils, class mobility, inquisitive and free-thinking students. Their ideas have successfully embedded themselves in the state sector, but failed disastrously in delivering on these promises. Instead we have been left with the bitter reality of failing schools where appalling behaviour is shrugged away as unremarkable. 

In 2000, the PISA index of student attainment was developed to make international comparisons in science, literacy and maths. Britain's slide down the rankings has been precipitous: we are now ranked 25th for reading and 28th for maths worldwide (compared with seventh and eighth respectively in 2000). In Britain, one in five pupils leaves school functionally illiterate, an unforgivable statistic for a country of our wealth and resources.

The Left's dedication to education cannot be faulted, but the influence of its ideas on the culture of education has been a disaster. Every day I see my school failing to educate its pupils, and I despair at how we teachers must stagger under the burden of bad ideas. It is time to abandon the damaging notions which have dominated educational thinking for the last half-century. In their place, we should welcome the return of discipline. In my history classroom in an inner-city comprehensive, I will be doing my bit to turn the tide.