A Peer-Reviewed Assessment of the MAST Methodology

Extracted text from MAST_Ethiopia_Pilot_Project_assessment_2015_-_observers_report.pdf:

A Peer-Reviewed Assessment of the MAST Methodology Ethiopia Pilot, August 2015 From August 3-15, 2015, a collection of Bible translation consultants, churches, facilitators, and indigenous translators assembled in Hawassa, Ethiopia to pursue a translation effort for five minority people groups that, to date, had yet to possess any Scripture in their heart languages. The idea for this project was initiated in March 2015 by Wycliffe Associates (WA) and the Maclellan Foundation, in primary partnership with the Ethiopian Kale Heywet Church, Wycliffe Ethiopia/Africa, The Word for the World, and The Seed Company. The objective of this translation pilot was to document the effectiveness of the process and delivered end products of the Mobilized Assistance Supporting Translation (MAST) methodology. This approach offered the potential for significant acceleration in the translation process with no loss of quality, at lower cost, in close partnership with local churches and therefore, was of the utmost interest to the Foundation. The intent was to pre-agree upon a set of quality, quantity, and process objectives, and then capture the delivered results after a two week pilot. Further detail on the pilot design is included on the following page, including the specific members of an Assessment Team of Bible translation consultants tasked with evaluating MAST. It is important to note that WA agreed to the initial four individuals selected to serve as evaluators. Later in the process, we (Maclellan) added a fifth consultant as a potential substitute because one of the initial four was experiencing health challenges. WA was not in favor of adding a fifth consultant, but graciously acquiesced for the sake of the project. The following document (pages 3-21 herein) provides the unified conclusions and recommendations reached by all five members of the Assessment Team. This unity is compelling when one recognizes the five consultants were from five different countries, represented four Bible translation organizations, and in aggregate, had over 100 years of experience in Bible translation work in Africa. The Foundation believes the report represents a balanced, constructive, accurate, and independent review of what the MAST methodology delivered in this specific Ethiopian context. Due to the nature of MAST, results may vary based on the languages and geographies engaged, and therefore, this review should not necessarily be interpreted as a blanket assessment of other MAST implementations. Neither should it be interpreted as an evaluation of broader issues, like organizational health or strategies. The report represents the first-hand evaluations of Bible translation consultants. It does not capture the opinions of the indigenous church that participated in the project. As a whole, the partnering local church expressed great enthusiasm for MAST leading into the pilot, and that enthusiasm remains the same post-pilot. While none of the teams were able to complete initial drafts of all three of the targeted books, the progress made was noteworthy when one considers the lack of training and literacy-levels of some of the indigenous translating teams.

Wycliffe Associates has requested additional detail to substantiate some of the summary perspectives communicated by the Assessment Team; specifically, comments made in Appendix 2 of the report. This is a perfectly valid request, and we are attempting to honor it. Below is the mutually agreed-upon design criteria: MAST Ethiopia Pilot Design Three NT books targeted for completion over a two-week period: Mark, Romans, and Galatians; translation work to be sequenced in that order; A defined set of pilot criteria, with a particular focus on quality standards, developed and approved by the translation assessors with Wycliffe Associates leadership; finalized in early July; Five translation teams comprised of 5 to 9 native speakers in each language: o o o o o Inor, a Gurage dialect (280,000 speakers) Baiso (5,500 speakers) Gayil (55,700 speakers) Dime (10,700 speakers, no writing system) Chara (13,000 speakers, an oral translation);

Translators from an array of churches, primarily within the Kale Heywet denomination. Translator literacy, familiarity with Amharic, and education levels varied widely; One Wycliffe Associates facilitator, and one or more Ethiopian translation consultants per team; Five member assessment team: Katy Barnwell and Ralph Hill (The Seed Company), Manie van den Heever (The Word for the World), Daniel Hankore (Wycliffe Ethiopia), and Paul Kimbi (Wycliffe Africa).

We honor and applaud all those who work each day to get the Scriptures into the hands of Gods children. We thank you for your service and sacrifice. Blessings,

David Denmark Executive Director, The Maclellan Foundation

Observations on the MAST Ethiopia Pilot Project and the MAST approach to Bible translation The Ethiopia Pilot Project was held in Hawassa, Ethiopia, from August 2-15, 2015

Outline

1. Executive Summary 2. Assessment of the Ethiopia pilot project and its outcomes 2.1 Work completed during the workshop 2.2 Observations on how the MAST program was applied 2.3 Assessment of translation quality 2.4 Assessment of the success of the project 2.5 Comparative assessment 3. Values and distinctives of the MAST process 4. Areas of concern, with some suggestions 5. Recommendations and suggestions 6. Seeking ways to work together more closely Appendix 1 Detailed observations on the application of the MAST process Appendix 2 Detailed assessment of translation quality based on agreed pre-pilot criteria

2. Executive Summary We commend warmly the enthusiasm and commitment of the WA team members. Their heart for getting Scriptures out at all costs to those who currently have no access to Scripture, is exemplary and warms our hearts too. Our longing is to work together to see the WA team use their commitment and their resources to make the greatest possible contribution to Bible translation worldwide. We appreciate and identify with the emphasis on reaching language groups of less than 10,000 speakers, which they sometimes refer to as the least of these (Matthew 25:45). The time is now ripe for reaching such groups. The pilot project observed in Ethiopia brought together five translation teams, who worked together in their first experience of Bible translation with enthusiasm. Four of the teams achieved at least a handwritten draft translation of the Gospel of Mark to varying degrees of completeness see further details below. This is a significant achievement. However, objective evidence demonstrated that the drafts are still needing considerable further work to improve accuracy and other qualities. Also, orthography issues and consistency of spelling still need to be reviewed and the texts need to be keyboarded. In order for MAST programs to achieve acceptable draft translations of a quality ready for testing and reviewing by churches and communities, the following seem essential: (a) foundational preparation, as described in detail in section 4.1, and (b) basic and ongoing training in translation skills for the participants.

Page 1 of 19

The initial training in translation could be done in one of two ways: (a) by holding a training course in advance of the first MAST workshop, or (b) by including training at relevant points within the workshop itself. Application to the book in focus would then reinforce these principles and guide the translators in developing good translation habits. It would seem essential also to include training to equip the teams to continue working after the initial workshop. The intention is that they will work closely with the churches in their home areas to complete the testing, reviewing, and formatting needed to bring the draft translation to publication, whether in audio, print, and/or digital format, and to see the translations launched and in use. The national Bible translation organization would also be contributing to this process. This training in translation skills might initially seem to slow the program but would result in much improved draft translations and increased speed and quality in the long run. In Ethiopia, in at least some projects, time pressure was also a factor which affected the quality of the translation drafts. The translators were sometimes heard to express a felt need for more time to finish a given step when being encouraged to move on to a new step. It must also be noted that the 8 steps which are central to the MAST methodology were not always rigorously applied. In order for the necessary foundational steps to be completed before translation begins in a project, the MAST team needs to work closely with a local organization, usually a national Bible translation organization, who can be responsible for these activities. This would build on what has already been done in the region concerned, using the available knowledge of the local situation, and relationships with the church and other partners that are already in place or under development. National Bible translation organizations function as an arm of the church, serving the church through Bible translation. They can also help to provide the essential training in translation principles and in ongoing procedures. We therefore recommend that, wherever possible, both the goals and methodology for any MAST program should be discussed and decided jointly by the national partner interacting with the MAST leadership, in order to be appropriate for that situation. Where such partnership can be established, the output of MAST workshops could be considerably improved, both in quality and in quantity. The name Mobilized Assistance Supporting Translation assumes a supportive role. In summary, the MAST program can provide valuable supportive assistance when it can serve to complement the work of a partner on the ground who is able to (a) do the necessary preparation work, (b) provide aspects of training that are not offered by MAST, and (c) facilitate the ongoing follow-up processing through to publication in the appropriate format (audio and/or print, digital or otherwise). MAST consists of a procedure, unfortunately one that shortcuts some steps necessary to achieve a good translation. Without the partnership of a Bible translation organization, the knowledge and translation expertise needed to help a local church or community translate the Scriptures are missing.

Page 2 of 19

In summary, the translation goals set by WA for the Ethiopia workshop (Mark, Romans, and Galatians in the five languages) were not reached. Moreover, the quality of the drafts was not such that they were ready for the next stage of church and community review. While recognizing the hard work and dedication of all involved, it has to be said that the rate of progress and the quality achieved clearly do not substantiate the widely publicized claims made for the accelerated rate of translation that can be achieved through the MAST methodology. An example of such claims is that it is possible for 26 speakers of a language to draft a complete New Testament in 2 weeks, with no previous translation training or experience. In section 5, we offer some suggestions on ways in which WA, working closely with other partners, could use available human and other resources to make a very significant and lasting contribution to Bible translation worldwide.

2. Assessment of the Ethiopia pilot project and its outcomes 2.1 Work completed during the workshop The stated goal was to achieve a draft of Mark, Romans, and Galatians through the verse-by-verse check, i.e., up to the stage that it would be ready to present to church leaders and the community for further testing, reviewing, and improvement. The teams achieved the following: Inor (Inor is the spelling of the name in the Ethnologue; alternative spellings are Enor or Enore) 8 participants: Achieved a first draft of Mark and Romans, up to and including verse-by-verse checking. The work on Romans was done very quickly during the last two days. The Inor language is a Semitic language, quite closely related to Amharic and with similar structures. The translators therefore found it fairly easy to write the language, because they were used to writing Amharic, and it seems they were able to achieve a fairly natural translation with minimal adjustment of form because of the structural similarity to Amharic. They were also an exceptionally hard-working team. Three members of the team had benefitted from four weeks of training with The Word for the World. Bayiso 5 participants: The Bayiso team completed a draft of the Gospel of Mark up to and including the key term check; they completed the verse-by-verse check up to the end of chapter 14. Some members of this team had previously completed four weeks of training with The Word for the World. Gayeil (Ethnologue entry Gayil) 5 participants: The Gayeil team completed a draft of the Gospel of Mark and the epistle to the Galatians through the verse-byverse check stage. Verse-by-verse checks were carried out by a facilitator Page 3 of 19

working with team members individually, each working through the chapter they drafted. Checks involving the whole team were not included. Dime 8 participants: The Dime team drafted the Gospel of Mark. They completed a verse-by-verse check, up to the end of Mark chapter 11. The verse-by-verse checks were done with the whole team working together; this engendered useful discussion as they considered the merits of alternative translations, which helped the team develop their awareness of translation principles. Chara 9 participants: As no orthography has yet been developed for Chara, the team drafted orally. They completed oral drafts of the first 8 chapters of Mark, including verse-by-verse checks. Drafting was begun on chapters 9 and 10. It was observed that Chara is closely related to the Gofa language and that for future translation work, the team could use the Gofa translation as a primary source text. This would be much easier for them than using Amharic, an unrelated language, as their primary source. Each language team was assigned one WA facilitator, whose task was to guide the teams in applying the 8-step methodology, and two Ethiopian facilitators, staff members of Wycliffe Ethiopia or of The Word for the World Ethiopia, with varying levels of translation experience, who participated in the verse-by-verse checking process. The WA facilitators were not trained in translation principles.

2.2 Observations on how the MAST process was applied Eight steps were outlined for the MAST process. These were presented in an opening session by Dan Kramer and comprised: (a) 4 steps for drafting, listed as (i) consume the text, (ii) discuss the text with one other speaker, (iii) chunk the text, and (iv) blind-draft the text, and (b) 4 steps for checking, namely (i) self-check referring back to the source text, (ii) peer-check with one other speaker, (iii) key term check also involving a facilitator or advisor, and (iv) verse-by-verse check. A further step was mentioned, namely keyboarding or recording. Recording was done for the Chara oral project but not for other projects. There was no provision for keyboarding the draft translations. Other steps were listed in a paper that was distributed (headed MAST) but were not included in the Ethiopia program. These eight steps were completed in form by each language team, and were checked as done. However, the steps were not always rigorously applied and the goals of each step did not always seem to be achieved. For further observations on the application of the 8-step methodology, see Appendix 1. It seems that insufficient explanation or demonstration was given on the purpose of each step and how to apply it, as was apparent from the number of omissions and errors that were discovered during the verse-by-verse checks. For example, the goal of steps 1 to 3 was that the Page 4 of 19

translator should be able to remember the passage so that he could put away the written source text and attempt to translate each chunk without reference to it. It could be frequently observed, however, that the translators did not seem to have understood the text well enough to be able to draft it from memory, and were looking constantly at the written source text as they drafted. It is strongly recommended that more attention should be given to this first step of studying and understanding the meaning of the passage to be translated, developing appropriate exegetical skills. Translators should have ready access to at least two reliable source texts, preferably one more literal version and one more meaning-based version, and also to translation helps on the passage. The availability of digital recording devices would also assist translators in the blind-drafting step a procedure often referred to in translation training materials as oral drafting. It would have been good to include some comprehension testing even with members of the team, to see how well the translation of a chapter done by one team member communicated to another team member who had not worked on that chapter. In other respects, the MAST procedural steps are similar to the recommended drafting procedures practised in most Scripture translation projects see the attachment entitled 12 Translation Steps. Further observations on the application of the eightstep MAST procedure are listed in Appendix 1. It was challenging to track reliably which of the steps had been completed for the whole of each chapter.

2.3 Assessment of translation quality It was disappointing that the verse-by-verse checking stage (step 8) revealed a number of omissions and misunderstandings that should have been caught in the earlier procedural steps. Short sample consultant checks done after the verse-byverse check had been completed also revealed a number of errors, in some cases quite serious errors. While quality varied, the majority of the drafts were assessed to need more work before they would be ready to present to the churches and communities at home, for further testing and reviewing. To achieve an accurate translation it is essential that the translators thoroughly understand the message they are to translate. Such an understanding of the source text can only be achieved by developing an awareness of the historical and cultural background of the original message. In the Ethiopia workshop, however, participants translated directly from a single version of the text without any reference helps. Some misunderstandings were therefore inevitable. For more detailed observations, see Appendix 2.

Page 5 of 19

2.4 Assessment of the success of the project As is usual in translation workshops where teams from different languages work alongside each other, there was a good spirit of camaraderie and mutual encouragement. The participants appeared to enjoy their work! This was significant as the program was very good for building a team spirit and confidence in being able to do the job. When asked, Are you happy with what you have done? the reply was invariably yes. During the workshop, assessment of success often seemed to be based on rather general statements as to whether the translators or national advisors felt good about the process or the work being done. While the observers rejoiced in the good spirit, we would have been more encouraged if it had been apparent that the participants had in fact learned the skills needed to do the job well in the future and had been achieving quality translations.

2.5 Comparative assessment relative speed and quality of end products compared with those of other Bible translation projects A draft was completed on paper, but as noted elsewhere, the last check (verse-byverse check) indicated that the translation was not ready for sharing with other speakers, i.e., for the next step of testing for comprehension and other qualities. There were essential issues that had not yet been addressed. For example, orthographies had not been established and no attempt had been made to train participants to follow an agreed spelling system. No keyboarding was done as there was no team member with the necessary computer skills or training. Dialect issues had also not been considered. To compare the rate of drafting typically achieved in a MAST project with the rate of drafting that can be achieved in other situations in a workshop context, where (a) the translators have received at least basic training, (b) appropriate resources are provided (good source texts, Bible dictionary), and (c) the translator or another team member is able to keyboard the translated text, it is estimated that a good draft translation of Mark can be achieved in a four-week period by a new team. One-book workshops, often involving a cluster of related languages, are not uncommon in the translation world and could be promoted further. In summary, a first draft was achieved faster in the two-week Ethiopia MAST workshop than is typical in other initial translation workshops, but the quality was such that much more work will be needed to bring it to an acceptable quality and in a condition ready for wider testing.

Page 6 of 19

3. Values and distinctives of the MAST process 3.1 The marked focus on recognizing and promoting the responsibility of the local churches in the region concerned is strongly affirmed. In recent years, working in partnership with the local churches has been a central value for Bible translation organizations, as has also been the recognition that the translation must be owned by the language communities concerned. Ongoing interaction with local leaders needs to be the responsibility of the national Bible translation organization partner because of factors such as accessibility, ongoing contact and relationships, and the wider language of communication. These interactions will help to make ownership of the project by the churches viable and effective. There are two factors to consider: (1) There is a need to ensure that, in any given area, all the denominations in the area are involved to the maximum extent possible. (2) When a Scripture translation project is in view, a Bible translation organization should be one of the partners involved, in order to provide the necessary training and guidance on the actual translation work. Where there is no Bible translation organization in the region concerned, we recommend that the MAST team recruit a qualified translation consultant to work with them.

3.2 Focus on clusters of languages is another positive approach promoted by MAST. This method has also been experimented with and implemented in recent years by several Bible translation organizations, especially where there is potential for several related languages to be grouped together and to share in training and other activities. Cooperation in this kind of project is strongly encouraged.

3.3 The disciplined approach within the WA team is exemplary, respecting the values of humility and obedience, which they have identified as key values in their relationships within the team.

3.4 The workshop approach, focusing on the drafting and/or checking of one or more specific books, provides an effective opportunity to guide and support translators in developing their exegetical skills, a strategy also widely practised by other translation organizations. For example, when a team is about to translate their first epistle, it is ideal if they can participate in a workshop for that epistle, in order to equip them to meet the new challenges of translating this genre of text.

Page 7 of 19

3.5 Involving larger numbers of speakers on translation teams is certainly something to be considered. However, dividing books by giving one chapter each to different translators can only result in major inconsistencies in style and terminology, especially where they are working from different source versions. An alternative strategy might be as follows: Consider training an initial team, then developing three (for example) teams, with a trained translator as the leader of each team, training others, and then assigning different books to each team or individual translator. Assignment of responsibilities could be fitted to the strengths of the team members. This is something that could usefully be explored further, and could be particularly helpful for projects where the translation of the full Bible is the goal. This could be a key innovatory contribution of the MAST approach if applied carefully and creatively.

3.6 Development of translation apps to help those involved in translating materials or in other aspects of translation. Wycliffe Associates are to be warmly commended for the app they have developed, called translationStudio. This app provides a location in which to enter the translation, with the facility to access relevant helps when these are available. As currently developed, it is well set up for keyboarding translations of the Open Bible Stories compilation. When it comes to Bible translation, however, it is unfortunate that this app only includes the source texts developed by Wycliffe Associates (ULB and UDB in English, for example). See 4.3 for a discussion of WAs own source translations and 4.4 for a discussion of copyright and unrestricted licensing issues. For Scripture translations, the Paratext software is much to be preferred, providing excellent keyboarding, editing, and formatting functionality, and a huge range of consistency checking options as well as access to many Bible translations and translation helps.

4. Areas of concern, with some suggestions 4.1 It is important to recognize the essential preparatory steps that need to be taken before translation begins in a new project, and to work with partners to expedite these steps. This may appear to slow down work in the early stages, but if it is not done, there are likely to be serious delays or disagreements later. Beware of taking shortcuts that make the task longer in the end! Such shortcuts may give the impression that the goal is being achieved faster, but they inevitably result in loss of quality. Essential preparatory steps include:

Page 8 of 19

4.1.1 4.1.2

Conducting a language survey to identify use of language(s) in the region concerned and dialect differences. Building and promoting shared vision and cooperation between representatives of different denominations and other interest groups represented in the area. Working with church and other leaders in the area to establish a representative committee (with title appropriate to the situation). Discussing with the committee what the goals of the proposed project should be and what kind of translation they would like to have, who they anticipate will be the primary users of the translation, and also for what particular purposes it will be used. Investigating the available potential source texts that could be used in the process of translation, and discussion with church leaders on which of these should be the primary reference texts. This could include versions in major regional languages and other related languages which could be valuable source texts for the proposed new translation(s). Ensuring that sufficient copies of relevant versions and translation helps are available for use during the proposed workshop and for ongoing translation work. If not already in place, developing a consistent orthography, that has been accepted by the local community and approved by the authorities, as appropriate to the area. Using modern techniques for language analysis, with access to information about the sound systems and orthography of related languages, and in partnership with mother-tongue speakers, basic orthographies can be developed and tested within a three- or four-week period these days. Training the team members to use the agreed spelling system consistently; this saves time-consuming work on spelling correction later on. Training at least one or two members of the team in the use of a computer and appropriate translation-related software, such as Paratext, or identifying someone who already has that skill.

4.1.3 4.1.4

4.1.5

4.1.6

4.1.7

4.1.8 4.1.9

It is recognized that these procedures are not always possible in sensitive, unreached areas, and creativity is always encouraged in order to be prepared to work with less than the ideal.

Page 9 of 19

4.2 Provision of facilities and training to keyboard drafts from the start of translation work in each language During the Ethiopia workshop, the translators worked on handwritten drafts, using loose sheets of paper. By the time several steps had been completed there were often a number of corrections overwritten on the draft. (In one language, drafts were written with single line spacing because the need to leave space for changes and possibly back-translation was not mentioned.) This meant that when team checking was done, only one person, or at most two, could see the written text, and so other participants relied only on hearing the text read. When fresh handwritten copies were made, there was the danger of introducing new errors and also of confusion as to which was the latest updated version. In one case it was discovered that a sample consultant check had been done using a text that was in fact an earlier draft, not the text on which the verse-by-verse check had been done. This highlighted the potential for confusion and loss of valuable work. After the completion of the Ethiopia MAST workshop, considerable work will still be needed to keyboard the translations in the four written languages, and to get them into a condition where they can be shared with the churches. The Paratext software is designed to protect the team against such confusion, so that the master text is always clearly identified and can easily be updated. Printouts can be made for all the team for checking or reviewing. Projecting the translation on the wall so that all can see it is another useful technique to keep a group focused on the text, able to see proposed corrections as they are made, with the opportunity to approve or to express their alternative views. We recommend strongly that plans include provision for translation teams to use Paratext, a very valuable resource for Bible translation projects, providing ready access to source texts and helps for translation, as well as the facilities for keyboarding, editing, consistency checking, and preparation for typesetting and publication. Licences for programs such as Paratext and Translators Workplace are readily granted to all who will be using the helps contained in these programs in Bible translation. Finances may be an issue since there is a nominal charge for some programs. However, it seems there could be ways to resolve this through grants or donations.

4.3 Development of new source texts There are serious concerns over the WA program to prepare new translations in Gateway languages, which would then serve as source texts for languages in regions where these major languages are spoken. This decision seems to have been Page 10 of 19

prompted primarily by copyright and licensing issues; there also seems to be a misunderstanding that, where a copyrighted translation is used as the primary source text for a new translation, the organization that sponsors the new translation will somehow be in breach of copyright. It seems this is proposed even where there are already good translations in the major language concerned. In Amharic, for example, there are three translations: The Bible Society of Ethiopia 1962 translation (an older, more literal translation, perhaps comparable to the RSV), the Biblica translation (comparable to the NIV), and the BSE New Amharic translation (a common language translation, comparable to the Good News Bible). It would be a huge task to achieve new translations that would meet the expectations of accuracy and other qualities such that they would be trusted as source texts for further translations. The expertise needed for this would be better used in assisting translation projects in languages where there is no Scripture translation available yet, in the areas where these Gateway languages are used. In particular, we are concerned about the quality of the Unrestricted Literal Bible (ULB) and Unrestricted Dynamic Bible (UDB) that we understand would be used as the base texts for the proposed Gateway versions, and also as the primary source texts for those using English as their source language1. The knowledge that such versions were not based on translations published by the Bible Society or other reputable Bible publishers, could also undermine confidence in the translations that are based on these texts. We encourage WA to concentrate on facilitating translation work in languages where no Scripture is yet available, rather than promoting new translations for languages that already have published translations. There is a similar concern about the production of new sets of exegetical helps for translators. See section 4.4 below.

4.4 Development of sets of exegetical helps for translators This concern also applies to the development of new helps for translators, both in English and in other major source languages. It is painful to see new helps being prepared when excellent helps are already available. The expertise that is being incorporated to prepare these helps could rather be directed to fill the existing gaps, for example helps for those Old Testament books for which none are currently available.

1

Although Bible translations in languages of wider communication are often the immediate source for translations in minority languages, translations should, of course, be based ultimately on the original Hebrew, Aramaic or Greek text referring back to the original text when questions arise.

Page 11 of 19

It seems there have been difficulties in gaining access to such materials and making them available. We strongly recommend to all Bible translation organizations that we work together to overcome these obstacles. Recognizing the concern that WA has that there should be freedom to share these helps, we urge that the value of unrestricted licensing of such helps and training materials be seriously considered. This would mean that, while copyrights remain intact, translation resources produced by those organizations could be shared without restriction. In some cases, copyright of materials used within these helps is also a factor. We believe there may be creative ways to overcome this, possibly by modifying the format of such helps, or by providing links to the copyrighted material, instead of quoting them verbatim. WA has plans to produce a new set of translation helps for use in English and for adaptation into Gateway languages, making them accessible to translators who speak those languages as a second language. However, it seems likely that organizations such as the Bible Societies, SIL and the Seed Company, that have already produced excellent helps, may be willing to grant permission for translation and adaptation of these helps into Gateway languages. There is a difference between giving permission for the reproduction of materials which are already available, and giving permission for translation of such helps to make them available to those who need them in another language. We urge that all Bible translation organizations work together to coordinate the production of quality helps for translators and their adaptation into the relevant Gateway languages. These helps could then be made available on Paratext or Translators Workplace so that translators can use them together with the appropriate source texts. See also section 4.2.

4.5 Concerning the choice of participants (prospective translators) for MAST workshops and ongoing projects It was reported that in some MAST programs preliminary research was done concerning such factors as knowledge of the language of wider communication (LWC), literacy level, and right- or left-brain orientation of each participant and that teams were grouped accordingly. This was not done in the Ethiopia program, however. Experience has shown that it is wise to consider factors such as those listed in the previous paragraph, along with Bible knowledge and giftedness in expressing themselves in their own language. Other factors, such as the character and good standing of the individuals concerned within their community, also need to be taken into account.

Page 12 of 19

Having a larger number of participants on an initial workshop makes it possible to identify those who have good potential as translators so that longer-term assignments can be made accordingly. Clearly, churches in the language area need to be decision makers in the choice of translators, and care should be taken to see that all denominations in the language area are represented, as far as possible, on the team of translators and reviewers. 4.6 Training in translation principles and procedures The importance of the translators being aware of basic translation principles, such as the need to translate the meaning of the text, rather than maintaining the grammar, has been referred to elsewhere. This section includes further comments on this topic. The most important factor in achieving a good translation is the ability of the translator. Checking processes can catch some errors and help make some improvements, but the quality of the draft done by the translator is crucial. Therefore, issues such as (a) choice of the best available translators, (b) the initial and ongoing training of the translators, and (c) support through provision of adequate resources, as appropriate to the area, are of utmost importance. WA facilitators involved in the MAST process do not seem to be accessing, applying, or sharing the information in the Translation Manual available on Door43, which provides a basic introduction to translation principles. Draft translation helps that are being prepared for specific books by the WA and unfoldingWord team for specific books, such as those available for Romans and Galatians, were apparently not accessed or referred to during the Ethiopia MAST workshop, nor was any time scheduled for this. We strongly recommend that WA facilitators assisting on a MAST workshop should receive basic training on translation principles. Although the task assigned to them is currently to ensure that the 8 steps are systematically applied, they are often asked questions relating to translation procedures. 4.7 Choice of books to translate in early stages of the project The goals originally listed for the Ethiopian workshop were that the five teams would achieve drafts of Mark, Romans, and Galatians. It is a huge challenge for very new translators to jump from translating mainly narrative text such as Mark to the complex theological exposition found in Romans and Galatians. This was evident when a sample consultant check was done on the first few verses of Romans 8 drafted by a member of the Inor team. The translator used the more literal of the Amharic versions as his only source. In such a passage it would have been especially helpful to refer to the New Amharic version which unpacks the meaning, but the translator had evidently not yet learnt how to compare different versions, or indeed to think through the meaning of the text in his own language. Page 13 of 19

We recommend that translators be guided to begin by drafting mainly narrative texts, such as the gospels and Acts, then progress to easier epistles, such as James, and finally, when they have developed their skills, progress to the more difficult epistles. Similar guidelines apply for the translation of the Old Testament. 4.8 Cooperation in the development of translation consultants The focus on the acceleration of Bible translation work is warmly applauded. However, the rapid increase in the rate of starting new translation projects and producing translation drafts will create growing demands for translation consultant help. We urge that all organizations involved give attention to identifying those who have potential to develop as translation consultants, and take action to promote internship-type programs that will enable suitably qualified and committed translators to get the training and experience needed to develop in this way. In addition to helping to evaluate the quality of translations, translation consultants have an important contribution to make in training translators on the job, especially in the early stages of a translation project, and also in working actively with church representatives in the receptor language areas to approve translations for publication. Care is needed not to overload the national translation personnel (consultants, CiTs, and facilitators) in the long term, as they may already be carrying a heavy load of consulting and administrative responsibility in existing translation projects. 4.9 Expatriate presence in workshops In the Ethiopia Pilot project, a team of nine WA volunteers served as facilitators, with some other WA members also participating. In the Ethiopian situation, the number of expatriates was not a problem, especially as the workshop was held in a resort where there were foreign tourists also enjoying the facilities. These WA members and volunteers all contributed through their warm and caring participation. However, this number of expatriates might draw adverse attention in some locations. As a general principle, every effort should be made to involve trained nationals in filling appropriate roles, and to plan for ongoing national leadership. 4.10 Concern about relationships between organizations Wycliffe Ethiopia has expressed concern that some Bible translation organizations who come to Africa from the West seem to be competing with each other. This puts the national partners in a position of tension and may cause a divisive spirit among those involved in different translation projects because of financial support or other issues. As observers, we recognize this as a serious matter which needs to be addressed. We urge organizations to avoid claims that seem to be promoting themselves above other organizations.

Page 14 of 19

5. Recommendations and suggestions In this section, ways are suggested in which the MAST team could channel their human and other resources into contributing in very significant ways to Bible translation. This includes some activities that are already in process. The name MAST (Mobile Assistance Supporting Translation) could be very appropriately maintained to describe these activities. 5.1 Sponsor and support workshops, working with partner organizations WA can provide practical encouragement and help by sponsoring translation workshops working in close cooperation with partner organizations, who would be responsible for foundational preparation and for basic (introductory) training in translation principles and skills. National organizations would also provide trained consultants, consultants-in-training, and other facilitators to work with each team, helping them develop good methodology and also their awareness and ability to apply translation principles. Such workshops could help a cluster of languages get started by drafting a gospel, as some current MAST workshops do. Alternatively they might provide valuable help in situations where one or more teams have fallen behind their goals and need some stimulation and practical help. Such activities would be comparable to what WA has already been doing in sponsoring workshops over the last year, but with the agreement that the goals, translation procedures, and timing should not be a fixed package, but be determined with the national organization concerned and be set to achieve quality translations. 5.2 Preparation of Open Bible Stories WA have already made a very significant contribution by initiating and facilitating the preparation of Bible stories, especially more recently the Open Bible Stories compilation. Quite a high proportion of the MAST programs to date have had Bible stories as the goal. In some situations this can prepare the way for Bible translation. Awareness of cross-cultural and language transfer factors are still important for translating Bible stories; partnering national organizations can provide translation consultant help for Open Bible Stories projects. 5.3 IT support WA has strong IT personnel. We suggest WA consider forming a team of IT specialists who would become experts in the use of Paratext and other relevant software, as well as in training others in these programs. The team could then make field visits to run training workshops for prospective translators so that they become Page 15 of 19

proficient in the necessary computer skills and in using Paratext early on. They could also provide relevant technical (IT) help during translation workshops. In addition, such a team could provide help with keyboarding translations and stories, and checking for consistency in spelling and format, as has been done very effectively by MissionAssist (formerly Wycliffe Associates) in the UK. Further development and improvement of training materials for Paratext, including online modules, is also strongly encouraged and would be much appreciated. Another aspect of IT support could be to help teams get set up with Internet connections, and training them to use Skype or similar programs. 5.4 Provide training in English language proficiency, through workshops and online helps WAs experience in this area still has very useful potential. English language training could be geared specifically towards training translators in the use of translationrelated resource materials, such as using dictionaries and commentaries to find necessary information, and also in writing reports. 5.5 Develop programs to promote active involvement of volunteers in literacy activities and also in encouraging the use of translated Scriptures in creative ways WA could use their extensive volunteer base to make a huge impact in the area of Scripture engagement, which is an important part the whole Bible translation process. For example, they could assist in setting up listening groups in areas where new translations are coming into circulation. 5.6 Partnership in the development of Guest Bible Scholars (or name as preferred) who can help with exegetical checking of translations. WA are encouraged to elaborate plans to help volunteers who come with expertise in specific areas, especially biblical languages and exegesis, to develop the linguistic and cross-cultural awareness and skills to enable them to help check translations for exegetical accuracy. In some cases, when they have gained the necessary experience, they may eventually qualify as full translation consultants. Some organizations are already working to explore and develop this potential. They would welcome partnership and also be ready to help with orientation and training.

Page 16 of 19

6. Seeking ways to work together more closely May the God who gives endurance and encouragement give us a spirit of unity among ourselves as we follow Christ Jesus, so that with one heart and mouth we may glorify the God and Father of our Lord Jesus Christ. Romans 15:5,6 (adapted) We long to see all those involved in Bible translation, whatever their ministry or organization, working in close cooperation, so that all concerned can both benefit from, and contribute to, other work in progress. WA members represent many relevant areas of expertise, fresh insights, enthusiasm, and other resources. They have much to contribute! May God guide us to find ways to work together effectively.

______________________________________ Appendix 1 Detailed observations on the application of the MAST methodology in the Ethiopia workshop -- see 2.2 for a description of this process Concerning source texts: On the first day, printouts of one Amharic source text were distributed, without comment or discussion. Participants each received a printout of the chapter they were assigned to translate; there was no discussion on understanding the text and translating the meaning, so it seems they assumed they had to translate this literally. In order that translators acquire a good understanding of the message they are translating, it is important that they study the passage to be translated in more than one version, preferably at least one fairly literal version and one more meaning-based version. It would also have been helpful to have an overall introduction to the book to be translated its historical background and context, the audience for whom it was originally written, its structure, and main theme. Concerning knowledge of the source language: The participants in the Ethiopia workshop varied greatly in their understanding of Amharic, in literacy skills, and in Bible knowledge. Since participants were assigned each to translate different chapters, the quality of the drafts inevitably varied greatly. Concerning blind-drafting: It was observed that most participants did not in fact apply this procedure but worked directly from the written source text. Concerning the peer-check: The drafter read the text through with one other speaker. This did not always seem to include referring back to the source text for accuracy. Omissions or additions should have been noted at this stage but were often overlooked. Concerning chunking: The participants did not understand what they were asked to do and therefore this step was often missed or combined with other steps.

Page 17 of 19

The key terms check was generally not productive: It would have been helpful to provide an explanation of what a key term is, and of the factors that need to be considered in translating key terms, as well as alternative ways to translate key terms. A verse-by-verse check was in some cases done by the facilitator working with only the original drafter. It was suggested by the translators themselves that this check should be done with the whole team involved, and this did indeed give rise to good discussion, recognition of translation issues, and expression of different opinions. The opportunity for meaningful interaction between team members was missing in the early stages. The verse-by-verse check as applied in the Ethiopia workshop should not be confused with a consultant check. It would have been helpful if Ethiopian facilitators had been briefed on how to carry out the steps. These facilitators varied from two who were experienced and qualified consultants to those who had very limited translation experience. Expatriate facilitators were not trained in translation principles advice offered on translation issues often did not seem to be helpful and was sometimes even misleading.

Appendix 2. Detailed assessment of translation quality based on agreed pre-pilot criteria see also 2.3 Accuracy a. Faithfulness to the meaning of the original text: In all cases observed, the verseby-verse checks revealed some omissions and misunderstandings. We believe that more accurate translations could have been achieved if more attention had been given to helping translators understand the meaning of the source text well at step 1, and also to applying the other steps more rigorously. b. Completeness: Some steps were not completed to the end of every chapter because of time constraints. Participants occasionally expressed their concern at being asked to hurry and to move on to another step before they had finished the one on which they were working. c. Consistency: Consistency was not a value that was addressed, to our observation. Participants were assigned different chapters of the book to be translated. It seemed there was no attempt at coordination in their translation style or choice of key terms, for example. After day 1, translators on the same team were following different source versions -- some followed the Amharic 1962, some the Biblica version (NIV equivalent each participant received a copy of the chapter(s) they were assigned). For those working on Romans, a copy of the text of the BSE New Amharic Version (Good News equivalent) was provided.

Page 18 of 19

d. Key terms: As noted above, an explanation was needed of what a key term is, as well as of the factors to be considered in translating key terms, with information on alternative ways to translate. To give an example of one key term issue, in the Dime project, in the verse-byverse check, a consultant asked about the key terms in Mark 1:4 baptism of repentance for the forgiveness of sins none of these four key terms had been discussed by the team earlier in the process, and it seems that the term for forgiveness was missing. It seemed that neither facilitators nor translators kept a record of the proposed translation, or tentative translation, of the basic key terms in the Gospel of Mark, for example, in the language concerned. Clarity and naturalness Clarity can best be assessed by testing the translation with other speakers representative of the intended audience, asking them to retell what they understood. This will be a major focus at the next stage in the translation process. This process evaluates how well the draft is communicating to other speakers (of various categories) and also whether the meaning communicated is faithful to the original meaning. It was observed that participants were often finding it hard to read the translations drafted by others, or even their own translations. This seems to be evidence of problems of clarity and naturalness. Some idioms and collocations were translated literally from Amharic. For example, in one case, Some of you will not taste death was translated literally from the Amharic text; when questioned, the translators said the natural way to translate this would in fact be death will not touch some of you before . . .

These notes were compiled by Paul Kimbi (Wycliffe Africa Area), Manie van den Heever (The Word for the World), Daniel Hankore (Wycliffe Ethiopia), Ralph Hill (Seed Company) and Katy Barnwell (Seed Company), in response to a request to observe the MAST Pilot Project in Ethiopia. This review was conducted under the auspices of the Maclellan Foundation, with direction from Jim Wert and Randy Kennedy. Hawassa, Ethiopia August 2015

Page 19 of 19