OPERATOR’S TRAINING AT THE ESRF L. Hardy, JM Filhol ESRF, BP 220 avenue des Martyrs, 38043 Grenoble Cedex. Abstract The ESRF (European Synchrotron Radiation Facility) is an X-ray source of the third generation. The accelerator complex is composed of a Linear accelerator (e- 200 MeV), a synchrotron (300 meters - 6 GeV) and a Storage Ring (844 metres). Only one full-time Operator and one volunteer on shift (engineer or technician) operate these accelerators from the Control Room. A high level of versatility is required of each Operator. The Operation group training policy focuses on three methods. The first one is day-to-day informal dialogue between the Operators and the Machine experts. The second method is for each Operator to deepen his own knowledge of the various pieces of equipment and afterwards to transmit his knowledge to his colleagues. Finally, the usual internal/external training courses are considered. We will describe the ESRF Operation Group training policy (frequency and budget). PRELIMINARY In order to understand the training policy for Operators at the ESRF, it is necessary to explain some specificities of operation in our institute. Once this is clear, the limits for the level of training of the Operators will be determined. Who is working in the Control Room? An operating crew is composed of two people: The full time Operator and one other person working about 10 % of his working time on shift. During the weekend, holidays or nights, a third volunteer may be enlisted to reinforce the crew. These second or third members of the crew are either engineers or technicians who are involved in one way or another in the life of the Machine. This now concerns about 60 people with a very wide range of technical skills (from computing engineers or technicians to accelerator physicists and directors). This system enables as many members of staff as possible to participate in the life of the Machine and allows a mixture of fields of competence in the Control Room whilst always having a professional Operator on hand. In this way the volunteers participate actively in the life of the Machine (and hence feel more involved in the equipment they are responsible for). Another advantage is that the Operators can hold technical discussions with experts from various fields. The callable / stand-by system It is evident that during normal working hours the relevant people on site will be called immediately to solve any problems which may arise. However, outside normal working hours, a procedure has been established. Different levels of problem have been identified, the most serious of which being: - A failure on a piece of equipment and it is impossible to store the beam or even to refill the Machine. - The beam is stored but a failure on other equipment will prevent the refill of the Machine in the event of a beam loss. For these strategic pieces of equipment, different people are on ‘stand-by’ and are equipped with a beeper so that they can be reached 24 hours a day. These people must immediately react and go to the Control Room if the problem cannot be resolved over the telephone. More than ten pieces of equipment are covered by people on stand-by. Besides these groups, a group named ‘More Experienced Shift Leaders’ composed of people having a good knowledge of accelerator physics is also on stand-by. Their role is not only to intervene in problems which are not covered by the other groups, but also to provide help to the crews when needed, for instance to develop or confirm a diagnosis of a failure. SCHOLASTIC AND PROFESSIONAL BACKGROUND OF THE OPERATORS At the time the Operators were hired, it was the resolution of the Operation Managers to employ a broad range of Operators as far as qualifications, professional background and age were concerned. This is why, six years ago, the youngest Operator was hired at 26 whereas the senior of the team was hired at 42 years old. Globally, most of them have studied two or three years after high school. Their technical skills are various: some are more skilled in electronics, others in mechanics. The only common point between all of them is that almost none of them had seen an accelerator before coming to the ESRF! WORK SCHEDULE OF THE OPERATORS Although the ESRF Operators are professional full time Operators, they work 75 % of their time on shift. For more details about the way this is organised, see ref. [1]. This means that they can spend 25 % of their time on ‘Normal Days’, i.e., during normal working hours. This time will be used to attend training courses or meetings or to lead some useful short-term projects (not only for the Control Room but also for other groups). WHAT DO WE EXPECT FROM THE OPERATORS ON SHIFT? During their shifts, the Operators monitor many beam parameters (emittance, lifetime, etc) hence, are able to detect any abnormal drift or accident. They are responsible for carrying out the refills of the Machine and for the application of special procedures (for instance, a cleaning process must be applied for the time-structured filling patterns). They have the faculty to hold technical discussions with the Users who may call the Control Room for any explanation (“I see a beam drift or jump, is it due to an orbit drift or is it happening on my side?”, “I can not open my Front end, what are the possible reasons?”, etc). They are also asked to react quickly in case of an emergency on the ESRF site (fire, injury or similar situation) But one of the most important part of their job is the ability to make a good diagnosis when something abnormal happens. Let us take a simple example: in the middle of the night, there is a beam lifetime accident. This might be due to a problem of measurement and/or data processing from the current transformer. In this case, this can wait the next morning to be solved. If the beam lifetime accident is real it may have been induced by a failure on a Power Supply, which will require to call the people on stand by. As a consequence, the tune may be close to a resonance which, in this case, would require the advice of the ‘More Experienced Shift Leader’. This may be due to a serious pressure rise somewhere. In such a case, the Safety officer must be called in order to measure the radiation level close to the location of the pressure rise. The members of the Vacuum Group should be called to detect and fix the leak. This simple example demonstrates the crucial role of the Operator who must have a good overview of the way the accelerators are running! WE NOW HAVE ALL THE INFORMATION REQUIRED TO EXPLAIN THE TRAINING POLICY OF THE ESRF OPERATORS! In external training courses, the Operators may learn how a current transformer works, how a vacuum leak is detected and fixed, what are the physics behind the ‘resonance correction’ but, of course, they will never learn how to make a diagnosis with the ‘tools’ provided in the ESRF Control Room! This is why we firmly believe that the first way to train our Operators to be efficient on the spot is to initiate dialogues between the Operators and the experts who develop the equipment/software or with the technicians who upgrade and maintain the different equipment. You may wonder how it is possible to be so informally trained? This is possible due to the fact that, in addition to the Operator, the shift crew in the Control Room is composed of one or two volunteers who are these very engineers and technicians who build or maintain all the accelerator equipment. We consider this as a means for efficient internal on-the-spot training. OTHER IN-HOUSE TRAINING From time to time, i.e. about four times a year, some in-house training courses are organised for the Operators. They are generally initiated by experts who want to present some drastic modifications of their equipment for which there are repercussions in the Control Room (new software, etc). EXTERNAL TRAINING At the ESRF there is an open-minded policy of external training provided this is necessary for improved efficiency of the operator’s task. The Operators can benefit from this external training. However, as explained above, it is not the preferred training method. In order to have a good overview of accelerator physics, almost all the Operators have attended some training related to this subject. We choose the training courses as a function of their practical approach, avoiding too much mathematics. We consider, for instance, that a course at the level of the CERN School is not adapted to our Operators. Most of the Operators have followed training about computing systems. One of the most important parts of the Operator’s training is the English language since it is the working language at the ESRF! These external training courses represent about 8 % of the time spent in Normal Days (which themselves represent 25 % of the total working time of an Operator). Up to now, an average of 3500 FF (about 600 US $) per Operator is spent each year for their training. SAFETY TRAINING The operating crew has a crucial safety role in the Control Room. Indeed, at the ESRF, the CTRM is the only place which is manned 24 hours a day, all through the year. For this reason, the operating crew is in charge of initiating all safety actions. Concretely, safety is ensured at the ESRF in the following way: All around the ESRF there are hundreds of red telephones directly connected to the CTRM. In the event of an emergency (fire, accident) witnesses will automatically warn the CTRM. All of the fire detectors are also connected to a central station in the CTRM. A simplified procedure exists for each type of incident. The shift crew will apply the appropriate procedure. In some cases, the Operator himself will manage the whole intervention but, depending on the gravity of events, he could also request the help of a specialised intervention team composed of professional firemen and first-aid workers who are permanently on stand by. It means that the Operators must be trained for safety interventions. All of them have a first aid certificate and must follow a refresher course each year. In addition, our Operators are entitled to make radiation controls during the intervention: they have therefore received the corresponding training. OTHER TRAINING METHODS Visits to other centres Whenever this is possible, we try to visit other centres. Either we send two or three Operators to another accelerator centre for a few days (they have already been to Daresbury, UK and Elettra, Trieste, Italy) or we go all together to a centre close to Grenoble and visit other Control Rooms. For instance we went to the CERN, to a nuclear power plant and we plan to go to an Airport Control Tower. Meetings Each week, the Operation Managers hold a meeting to which all the engineers and technicians involved in the life of the Machine are invited. We review the previous week of delivery and discuss failures in detail with the goal of understanding the origin of the problems and the solutions which were applied. The Operators are of course welcome to attend this meeting since it provides another means of on the spot training. Every six weeks an Operator’s meeting is organised to exchange ideas and experiences between the Operators and the Operation Managers. This provides the opportunity for the Operation Managers to give some training and to develop subjects requested by the Operators (for instance: physical approach of the lifetime, basics of beam optics, etc) Exchanges between the Operators themselves Three years ago, an attempt was made to ask the Operators to choose a piece of equipment with the aim of writing an interactive documentation about it (on Hypercard software). Despite the fact that this was completely done for one piece of equipment, we must admit that the expected result was not reached. The reasons for this were three-fold : when there was a real failure on this piece of equipment, the first reaction was to call the person on stand-by and not to spend time in front of a screen. Secondly, the Operators were simply not delighted by the idea of spending hours in front of a screen in order to develop their technical knowledge. Finally the main underlying reason for the failure of this method was the complete lack of dialogue. From these conclusions a new, very recent idea has emerged. Each Operator will be asked to choose a piece of equipment that he will carefully follow. He will examine and deepen his understanding of all failures on this equipment, will learn how the failed sub-equipment works and will provide a report to the Operator’s meetings. We believe that this is a good way for the Operators to learn and teach their colleagues. CONCLUSION After six years of experience we are convinced that the best training method is the informal one where the Operators can dialogue with the experts who really know about the Machine. Meetings or exchanges of information between the Operators also seem to be an efficient, although indirect, way of training. From 1996 until now, the Mean Duration of a failure is one hour (with a Mean Time Between Failure of 35 hours). We do believe that this short duration can mostly be attributed to the Operators who have learnt to make a good diagnosis, which in turn enables them to intervene directly or at the very least, which enables them to call the right person who will solve the problem. Saving time is important when up to 40 beam lines are taking beam simultaneously. REFERENCES [1] L Hardy, JM Filhol, Operation at the ESRF: How we work, WAO 98