Planning and Reporting

Evaluation of the Women's Program

Volume I - Final Report



APPENDIX D
DATA COLLECTION INSTRUMENTS

File Review

Instructions:  Please restrict your review to the following key documents:

  • Proposal/Application form (AP)
  • Recommendation for approval (REC)
  • Follow up letter (sent after Minister's approval of funding) (LET)
  • Interim report (IR)
  • Interim report review (IRR)
  • Final report (FR)
  • Close-out assessment (CL)

Please note that standard forms were introduced in April 2003; before then, some of the documents may be in a variety of formats and the follow-up letter may not exist in some files. Using the above acronyms, indicate the source document where requested.  For information found in more than one document, indicate all sources, verify that information is consistent, and note any discrepancies in Question 33.



1. Name of organization: ______________________________________
2. Title of initiative: _______________________________________
3. Fiscal year (from database): ______________
4. Length of funding:
___ Less than 18 months
___ 18-36 months (multi-year)
5. Duration of initiative (FR)
From (mm/dd/yy):_____________ To (mm/dd/yy):_____________
___ Can't determine
6. Was the duration of the initiative extended?
___ Yes          ___ No         
___ Can't determine
7. If yes to Q. 6, to what date? (This should match final date in Q. 5)
(mm/dd/yy):__________________                ___ Can't determine
8. Date final report received (mm/dd/yy):_________________
___ Can't determine   ___ Report not submitted
9. Region:
___ Atlantic              ___ BC/YK
___ National             ___ Ontario
___ Prairie/NWT       ___ QC/Nunavut 
10. Priority funding area:
___ Improving women's economic status
___ Eliminating systemic violence
___ Achieving social justice
___ Other_________________________
___ Can't determine
11. Total cost of
initiative:

$____________
___ Can't determine
12. Funding
requested:

$____________
___ Can't determine
13. Funding
received:

$____________
___ Can't determine
14. Was supplemental funding received for this initiative?       
 ___ Yes              ___ No              ___ Can't determine

14a. Amount of supplemental funding  $_____________
15. The total amount of funding from sources other than SWC (REC budget) DO NOT include in-kind contributions or the initiative's own contributions.
$_______________
___ Not applicable, no other sources of funding
___ Can't determine
16. Please indicate the other sources of funding, if any. (REC budget) DO NOT include in-kind contributions or the initiative's own contributions.
___ Other federal departments         
___ Provincial government
___ Municipal government               
___ Foundation    
___ Other______________
___ Not applicable, no other sources of funding
___ Can't determine
17. Brief description of initiative (2 or 3 sentences). (REC - Section 12)
______________________________________
18. Primary group of women affected by the initiative. (REC - Section 13D and E)
___ Women in general
___ Lesbians
___ Women with disabilities
___ Aboriginal, Métis and/or Inuit women
___ Visible minority women
___ Immigrant/refugee women
___ Rural, remote, or northern women
___ Official language minority women
___ Low income
___ Young women (16-25)
___ Other____________________
___ Can't determine 
19. Which of the following Women's Program objectives does the initiative address? Cite sources.
___ Promote policies and programs within key institutions
___ Facilitate the involvement of women's organizations in the public policy process
___ Increase public understanding in order to encourage action on women's equality issues
___ Enhance the effectiveness of actions undertaken by women's organizations to improve the situation of women
___ Can't determine 

Source(s):______________________________________

Using the documents indicated below, complete the table below.  Objectives may have more than one expected outcome: use one line per outcome.

20. Objectives and outcomes of initiative
Objectives
(AP, REC, LET)
Expected outcomes
(AP, REC, LET)
Outcomes achieved
(IR, FR and CL)
    ___ Yes, fully  ___ Yes, partially  ___ No 
___ Can't determine
    ___ Yes, fully  ___ Yes, partially  ___ No  
___ Can't determine
    ___ Yes, fully  ___ Yes, partially  ___ No 
___ Can't determine
    ___ Yes, fully  ___ Yes, partially  ___ No  
___ Can't determine
    ___ Yes, fully  ___ Yes, partially  ___ No  
___ Can't determine
    ___ Yes, fully  ___ Yes, partially  ___ No  
___ Can't determine
    ___ Yes, fully  ___ Yes, partially  ___ No  
___ Can't determine
    ___ Yes, fully  ___ Yes, partially  ___ No  
___ Can't determine
    ___ Yes, fully  ___ Yes, partially  ___ No  
___ Can't determine
    ___ Yes, fully  ___ Yes, partially  ___ No  
___ Can't determine
    ___ Yes, fully  ___ Yes, partially  ___ No  
___ Can't determine
    ___ Yes, fully  ___ Yes, partially  ___ No  
___ Can't determine


21. Were the objectives reported in the AP and FR consistent?
___ Yes   ___ No   ___ Can't determine
22. Were the expected outcomes reported in the AP and FR consistent?
___ Yes   ___ No   ___ Can't determine
23. Were the expected outcomes linked to specific objectives?
___ Yes, all  ___ Yes, some  ___ No  ___ Can't determine
24. Did the FR report on whether expected outcomes (as opposed to objectives) were achieved?
___ Yes   ___ No   ___ Can't determine
25. Did the CL report on whether each of the expected outcomes identified in the FR were achieved?
___ Yes   ___ No   ___ Not applicable - no CL report (standard or non-standard)  
___ Can't determine
26. Did the initiative report that its objectives, activities, or budget changed during the funding period?
___ Yes   ___ No   ___ Can't determine

26a. Describe and note reason for change.  Cite sources.
_____________________________________________
_____________________________________________
_____________________________________________
_____________________________________________

Source(s):__________________________________________
27. Did the initiative experience any difficulties in meeting its objectives?
___ Yes   ___ No   ___ Can't determine

27a. Describe any difficulties. Cite sources.
_____________________________________________
_____________________________________________
_____________________________________________
_____________________________________________

Source(s):__________________________________________
28. If the Close-out Assessment Form was used for this initiative, indicate the outcomes from the Women's Program Accountability Framework that were met by this initiative.
___ Not applicable - form not used.
Institutional Change
___ Short-term outcome 1
___ Intermediate outcome 1A
___ Intermediate outcome 1B
___ Long-term outcome 1
Public Policy
___ Short-term outcome 2
___ Intermediate outcome 2A
___ Intermediate outcome 2B
___ Long-term outcome 2
Public Understanding
___ Short-term outcome 3
___ Intermediate outcome 3
Enhancing Effectiveness
___ Short-term outcome  4
___ Intermediate outcome 4
29. Does the file document any unexpected outcomes of the initiative, either positive or negative?
___ Yes, positive                   ___ Yes, negative                  ___ No

29a. Describe any unexpected outcomes.  Cite sources.
_____________________________________________
_____________________________________________
_____________________________________________
_____________________________________________

Source(s):__________________________________________
30. Does the Final Report on the initiative indicate plans to continue its activities in the funded area?
___ Yes           ___ No             ___ Can't determine

31. Please indicate if there appear to have been any difficulties in reporting.
___ No difficulties                          ___ National/regional office did not accept final report
___ Inconsistent information         ___ Missing documents
___ Failure to fulfil reporting requirements (substantively lacking).
___ Incomplete documents

Describe.
_____________________________________________
_____________________________________________

Other. Describe.
_____________________________________________
_____________________________________________

Can't determine.
_____________________________________________
_____________________________________________

32. For 2003-04 files, check the documents that were included in the file:
___ Proposal/Application form (AP)
___ Recommendation for approval (REC)
___ Follow up letter (LET)
___ Interim report (IR)
___ Interim report review (IRR)
___ Final report (FR)
___ Close-out assessment (CL)
33. Note any difficulties with reviewing file (missing documents, incomplete information, etc)
_____________________________________________
_____________________________________________
_____________________________________________
_____________________________________________
_____________________________________________
_____________________________________________
_____________________________________________
_____________________________________________

34. Additional comments:
_____________________________________________
_____________________________________________
_____________________________________________
_____________________________________________
_____________________________________________
_____________________________________________



Evaluation of the Women's Program
Interview Guide for Applicants
Background
  1. Please tell me about your organization. When was it established? What is its mandate? 

  2. How familiar are you with the Women's Program (WP)?
Design and delivery
  1. Has your organization received any non-financial assistance from the WP? If yes, what types of assistance has it received? (Probe: referrals to other funding sources, help in developing an action strategy, advice on operational concerns such as incorporation or financial administration, assistance with proposal preparation)

  2. How useful was the assistance you received? What difference did it make to your organization?  Is there anything else that would have been useful?

  3. Has your organization ever received funding from the WP?  (Probe: Was this prior to or after 1998?  If prior to 1998, did your group receive program or project funding?  If after 1998, has your group ever received 18-month funding? Multi-year funding?)

  4. Has your organization ever submitted a written request for funding that was not approved?  Did program staff explain the decision and your options? Do you have any concerns about the way your funding request was handled?

  5. Thinking about your organization's experience in developing proposals for WP funding, overall, how would you describe the process?  (Probe: Is it straightforward or onerous, and why?)

  6. Do you have concerns about any aspects of the proposal development/approval process?  (Probe any concerns with respect to: clarity of eligibility criteria, proposal assessment criteria, or approval process; assistance provided by staff; staff communication of expectations). What suggestions do you have for improving the proposal development/approval process?

  7. As you may know, the WP has evolved over the years.  How has each of the following changes affected your organization's ability to work on advancing women's equality?  (Probe: Have the changes been positive or negative, and why? How, if at all, has your level of funding been affected?)

    • Termination of program funding
    • Introduction of initiative funding
    • Availability of 18-month funding
    • Availability of multi-year funding

  8. As you may know, the WP introduced its most recent change in 2003, when it implemented an outcome-based approach that focuses on measuring results achieved by initiatives. What has been the impact of the outcome-based approach for your organization?  (Probe: Ability to identify outcomes and indicators; plan effectively; demonstrate results)

  9. Do you have any concerns about the outcome-based approach? What could the WP do to make it easier for your organization to apply the outcome-based approach?

  10. In your opinion, how could the WP be improved? (Note to interviewer: Ask for specifics)
Impact
  1. What has WP funding or assistance enabled your organization to do?  What has changed as a result of these activities? (Note to interviewer: Make sure interviewee distinguishes between activities and outcomes)

  2. Besides the WP, what other factors might have contributed to these changes?

  3. Has the WP had any unexpected impacts for your organization, either positive or negative? 
Cost-effectiveness/Alternatives
  1. Does your organization receive funding from other sources? If yes, what is the relative importance of WP funding for your organization?  How is the WP different from these other programs?  How is it similar?
     
  2. Could the WP be better delivered by an organization other than Status of Women Canada? If so, by whom, and why?

  3. In your opinion, is there anything that Status of Women Canada could do that would be more effective than the WP in advancing women's equality?  Is there anything that Status of Women Canada could do in addition to the WP to be more effective in advancing women's equality?
Relevance/Rationale
  1. Is the WP still relevant to women in Canada? Why or why not?
     
  2. What would be the effect on your organization if there were no WP?  In general, what would be the effect on women's groups and other equality-seeking organizations? On the advancement of women's equality in general?

  3. Do you have any other comments?

Evaluation of the Women's Program
Interview Guide for Program Managers and Staff
Background
  1. Please tell me about your role in the Women's Program (WP). (Probe: How long have you been involved with the WP? With gender equality policy and programming more broadly?)
Design and delivery
  1. What are the strengths of the Women's Program?  Please consider the design and delivery of both its funding activities and its non-funding activities.

  2. Are there any fundamental weaknesses in the design or delivery of the WP? Again, please consider both its funding activities and its non-funding activities.

  3. Is the WP delivered efficiently?  How could its efficiency be enhanced? Please consider the following aspects of program delivery:

    • Proposal development/approval process
    • The community's access to WP staff
    • Timeliness of grant approval process
    • Staff access to appropriate technologies

  4. Do you have the necessary supports to do your job well? What would help?

  5. As you know, the WP has evolved over the years. How have the changes affected you as program staff?  How have they affected the women's groups and other equality-seeking organizations with whom you work?  Please consider each of the following changes:

    • Termination of program funding
    • Introduction of initiative funding
    • Introduction of 18-month funding
    • Introduction of multi-year funding
    • Introduction of the outcome-based approach (including logic model, new reporting requirements, and new forms)

  6. How could the design and delivery of the Women's Program be improved? (Note to interviewer: Ask for specifics)
Impact
  1. Since you have been involved with the WP, what progress has been made toward achieving the program's four objectives, i.e.:

    • Promoting institutional change? 
    • Influencing public policy?
    • Increasing public understanding?
    • Building capacity of women's groups/other equality-seeking organizations?

    Can you give some examples of changes in these areas? To what extent do you think the WP contributed to these changes?

  2. What other factors might have contributed to these changes?

  3. Since 2000-2001, the WP has received $2 million in additional funding per year as part of the Agenda for Gender Equality (AGE) initiative. Based on your observations, has this additional funding allowed the WP to:

    • Broaden the scope of groups receiving support?
    • Increase its ability to support action on new and emerging issues?

    Please give specific examples for each to support your opinion.

  4. In your opinion, to what extent has the WP helped to achieve the strategic objectives of SWC?  (Note: strategic objectives for SWC in 2004-05 are strengthened and more equitable public policy; a broader range of informed stakeholders; increased departmental effectiveness)

  5. Do you think the WP has had any unexpected impacts, either positive or negative?

  6. In your opinion, are the expected outcomes of the WP clear enough to measure? How could the expected outcomes be clarified?

  7. In your opinion, is the new performance measurement strategy capturing adequate and useful information for assessing program impact?  (Probe: This includes both the reporting system and the administrative database). To your knowledge, have all regions and the national office implemented the strategy?  If not, why not?  How could the performance measurement strategy be improved?
Cost-effectiveness/Alternatives
  1. Are you aware of alternative models of program delivery that would be helpful for the Women's Program to consider? (Probe especially funding mechanisms used by other funders)

  2. In your opinion, is there anything that SWC could do that would be more effective than the WP in advancing women's equality?  In achieving the strategic outcomes of SWC?

  3. How is the WP different from other programs to advance women's equality in Canada?  How is it similar?  Is there any duplication with other programs? Does the WP work at cross-purposes with any other programs?

  4. Could the WP be better delivered by an organization other than SWC? If so, by whom, and why?

  5. In your opinion, is the WP giving Canadians value for their tax dollar?
Relevance/Rationale
  1. Is the WP still relevant to women in Canada?  Why or why not?  What would be the consequence for women's equality if the WP did not exist?

  2. How closely does the WP align with the current priorities of the federal government (i.e., from SFT, budgets, recent policy statements, etc.)?  What changes to the program would help to ensure better alignment?

  3. How closely does the WP align with SWC's strategic objectives? What changes to the program would help to ensure better alignment?  How could the WP more effectively contribute to the development of SWC's strategic objectives?

  4. Do you have any other comments?

Evaluation of the Women's Program
Interview Guide for External Stakeholders
Background
  1. Please tell me something about yourself and/or your organization. How are you involved in working on women's equality?

  2. How familiar are you with the WP?
Design and delivery
  1. In your opinion, what are the strengths of the Women's Program?  Please consider the design and delivery of both its funding activities and its non-funding activities.

  2. As far as you know, are there any fundamental weaknesses in the design or delivery of the WP? Again, please consider both its funding activities and its non-funding activities. (Probe: Give some examples of non-funding assistance if necessary).

  3. Is the WP effective at collaborating with external stakeholders such as other federal departments/agencies, other orders of government, community groups, and the private sector, to advance women's equality?  Please explain.

  4. In your opinion, how could the WP be improved?
Impact
  1. As you may know, the WP has four objectives: promoting institutional change; influencing public policy; increasing public understanding of action on women's equality issues; and building capacity among women's groups.  Since you have been involved with women's equality issues, what progress have you observed in each of these areas?  Can you give some specific examples?

  2. To what extent do you think the WP contributed to these changes? What other factors might have contributed?

  3. As far as you are aware, has the WP had any unexpected impacts, either positive or negative?
Cost-effectiveness/Alternatives
  1. Are you aware of alternative models of program delivery that would be helpful for the Women's Program to consider? (Probe especially funding mechanisms used by other funders)

  2. In your opinion, is there anything that SWC could do that would be more effective than the WP in advancing women's equality?

  3. How is the WP different from other programs to advance women's equality in Canada?  How is it similar?  Is there any duplication with other programs?  Does the WP work at cross-purposes with any other programs?

  4. Could the WP be better delivered by an organization other than Status of Women Canada (SWC)? If so, by whom, and why?

  5. In your opinion, is the WP giving Canadians value for their tax dollar?
Relevance/Rationale
  1. In your view, is the WP still relevant to women in Canada?  Why or why not?  What would be the consequence for the advancement of women's equality if the WP did not exist?

  2. Do you have any other comments?

Survey of Women's Groups and Other Equality-Seeking Organizations

Evaluation of the Women's Program

Status of Women Canada (SWC) has contracted PRA Inc., an independent research firm, to evaluate the Women's Program.

The evaluation is a requirement of the Treasury Board Secretariat of Canada for all federal grants and contributions programs. Its purpose is to examine the design and delivery, impact, cost-effectiveness, and relevance of the Women's Program. As this evaluation is being undertaken in the context of a government-wide Expenditure Review exercise, it is also designed to address questions of public accountability and alignment with the priorities of Canadians.
 
The views of women's groups and other equality-seeking organizations are very important to the evaluation. We would greatly appreciate it if a senior representative of your organization, together with whoever is most familiar with the Women's Program, would take the time to complete this survey.

All of your responses will remain confidential to PRA and will be reported only in aggregate. SWC will not have access to your individual responses. The administrative number appearing on the last page of this survey is to enable PRA to manage the survey process.

If you have any questions about the survey, please telephone Natalie Baydack of PRA toll-free at 1-888-877-6744. If you would like to receive an electronic (Word) copy of the survey on disk, please contact PRA.

Please return the survey in the envelope provided
or by toll-free fax at
1-800-717-5456

Your early attention to this survey is appreciated.
Please return the survey by
April 15, 2005


Part A:  Background information

This first section of the questionnaire gathers background information to help us understand the context for your responses.

  1. Is your organization a women's group or another equality-seeking organization? (Check one response only)
    ___ Women's group
    ___ Other equality-seeking organization

  2. What is the principal mandate of your organization? (Check one response only)
    ___ Public education
    ___ Advocacy
    ___ Research
    ___ Service delivery (e.g., counselling, training, shelter, etc.)
    ___ Other (specify) __________________________________

  3. What is the primary constituency of your organization?  We realize that you may serve more than one of the constituencies listed below. However, we are interested in the main constituency that you serve, so please check one response only.
    ___ Women in general
    ___ Lesbians
    ___ Women with disabilities
    ___ Aboriginal, Métis and/or Inuit women
    ___ Visible minority women
    ___ Immigrant and refugee women
    ___ Rural, remote or northern women
    ___ Official language minority women
    ___ Other (specify) ______________________

  4. What is the primary issue area that your organization works on? (Check one response only)
    ___ Economic
    ___ Violence
    ___ Social justice
    ___ Other (specify)______________________

  5. How long has your organization been in existence? (Check one response only)
    ___ Less than 1 year
    ___ 1-2 years
    ___ 3-5 years
    ___ 6-10 years
    ___ 11-15 years
    ___ 16-20 years
    ___ 21 years or more

  6. What is the scope of your organization? (Check one response only)
    ___ International
    ___ National
    ___ Interprovincial/interterritorial
    ___ Provincial/territorial
    ___ Regional
    ___ Municipal/local

  7. What is the approximate membership of your organization? (Check one response only)
    ___ Less than 20
    ___ 20-49
    ___ 50-100
    ___ More than 100
    ___ Don't know

  8. How familiar are you with the Women's Program?
    ___ Very familiar
    ___ Somewhat familiar
    ___ Not at all familiar


Part B:  Design and delivery of the Women's Program

Through the Women's Program, Status of Women Canada offers various types of support to women's groups and other equality-seeking organizations working in Canada to advance women's equality. This section of the survey asks you to consider the way the program is designed and delivered.

  1. To your knowledge, has your organization ever received any of the following types of assistance from Women's Program staff?
  Yes No Don't know
a. Help with developing an overall action strategy or plan for your organization      
b. Referrals or references to other funding sources      
c. Advice on operational issues, such as incorporation and financial administration      
d. Referrals to other resources in your community that might be of assistance to your organization      
e. Help with networking, communication, and partnerships between your organization and others      
f. Use of facilities and/or equipment (e.g., photocopier, meeting rooms)      
g. Issue/conflict resolution      
h. Identifying/providing relevant resources, documents or tools      
i. Other (please describe; do not include funding assistance)
__________________________________________
__________________________________________
     


If you answered "Yes" to any part of question 9, please complete questions 10 and 11.
Otherwise, please go directly to question 12.

  1. How would you describe the help you received from the Women's Program?
    ___ Very useful
    ___ Somewhat useful
    ___ Not at all useful

  2. Please explain your answer.
    __________________________________________
    __________________________________________
    __________________________________________
    __________________________________________

  3. The Women's Program also provides financial assistance to equality-seeking organizations for eligible initiatives, defined as "change-oriented strategies addressing one or more of the Women's Program objectives and areas of focus, which occur within a specified time-period and have clearly articulated plans to achieve results."  To your knowledge, has your organization ever...
  Yes No Don't know
a. Submitted a written request for funding that was NOT approved?      
b. Received funding from the Women's Program?      

Please answer questions 13 and 14 if your organization has ever submitted
a written request for funding that was NOT APPROVED.
Otherwise, please go directly to question 15.

  1. Please read each statement below and check the response that best represents your opinion.
  Strongly agree Agree Neutral Disagree Strongly disagree Don't know
a. Program staff made it clear why our organization's request for funding was not successful            
b. Staff provided information on alternative sources of funding for our initiative            
c. We were informed of how to have the decision on our funding request reviewed            
  1. Do you have any concerns about how your funding request was handled? If so, please explain.
    __________________________________________
    __________________________________________
    __________________________________________
    __________________________________________
    __________________________________________

    ___ No concerns

All respondents please answer questions 15 and 16.

  1. The next series of questions is about the proposal development/approval process. As you are aware, applicants are encouraged to contact Program staff early in the development of their funding idea so that staff can provide information, discuss issues, determine eligibility, assist in developing the initiative, and assess its potential for funding.

    Please read each statement below and check the response that best represents your opinion.
  Strongly agree Agree Neutral Disagree Strongly disagree Don't know
a. The eligibility criteria for funding are clear            
b. The proposal development process helped our organization to clarify the objectives of our initiative.            
c. The proposal development process helped our organization to identify appropriate strategies for our initiative            
d. During the proposal development process, our organization was required to demonstrate that we had involved a full diversity of women, affected by the issue, in our initiative            
e. Women's Program staff were clear in communicating expectations about proposal requirements            
f. I understand the criteria used to assess proposals for funding            
g. I understand how funding requests are approved            
  1. Do you have any suggestions for improving the proposal development/approval process?  Please explain.
    __________________________________________
    __________________________________________
    __________________________________________
    __________________________________________
    __________________________________________

    ___ No suggestions

If your organization has ever received funding from the Women's Program,
please complete questions 17 to 24.

Otherwise, please go directly to Part D.

  1. As you may know, the Women's Program has evolved over the years. Several major changes occurred in 1998:

    • Initiative funding replaced project and program funding. Initiative funding was intended to make the Program more equitable, flexible, and accountable.  It supports change-oriented strategies that address one or more of the Women's Program objectives and areas of focus and that have clearly articulated plans to achieve results.

    • Funding for initiatives up to 18 months in duration was made available. Prior to 1998, project funding had been available for up to 12 months only.

    • Multi-year funding was made available. Multi-year funding supports initiatives focusing on institutional and public policy change for up to three years in duration.  Prior to 1998, multi-year funding was not available.

  Yes No Don't know
a. Prior to 1998, did your organization receive program funding?      
b. Prior to 1998, was your organization ever refused program funding?      
c. Since 1998, has your organization received 18-month funding?      
d. Since 1998, has your organization received multi-year funding?      
  1. How have the 1998 changes to the Women's Program affected your organization's ability to work on advancing women's equality?  Please explain your answer in the space provided below each question.
  Positive change Neutral Negative change Don't know
a. Termination of program funding        
Please explain your answer.
__________________________________________
__________________________________________
__________________________________________
__________________________________________
__________________________________________
__________________________________________

  Positive change Neutral Negative change Don't know
b. Introduction of initiative funding        
Please explain your answer.
__________________________________________
__________________________________________
__________________________________________
__________________________________________
__________________________________________
__________________________________________

  Positive change Neutral Negative change Don't know
c. Availability of 18-month funding        
Please explain your answer.
__________________________________________
__________________________________________
__________________________________________
__________________________________________
__________________________________________
__________________________________________

  Positive change Neutral Negative change Don't know
d. Availability of multi-year funding        
Please explain your answer.
__________________________________________
__________________________________________
__________________________________________
__________________________________________
__________________________________________
__________________________________________

  1. The Women's Program introduced its most recent change in 2003, when it implemented an outcome-based approach that focuses on measuring results achieved by initiatives. At that time, a new Application Form and Guide to the Application Form were introduced, as well as Interim and Final Report Forms. Please read each statement below and check the response that best represents your opinion.
  Strongly agree Agree Neutral Disagree Strongly disagree Don't know
a. Women's Program staff have clearly explained the outcome-based approach to our group            
b. Assistance from Women's Program staff has helped our group to identify realistic outcomes and indicators            
c. As a result of the outcome-based approach, our group is able to plan more effectively            
d. The outcome-based approach has helped our group to demonstrate the results of our work to our members, to the community at large, and/or to other funders            
e. The Application Form and Guide are easy to understand            
f. The Interim and Final Reports are easy to use            
  1. What could the Women's Program do to make it easier for your group to apply the outcome-based approach?  Please explain.
    __________________________________________
    __________________________________________
    __________________________________________
    __________________________________________
    __________________________________________

    ___ No suggestions

Part C:  Impact of the Women's Program

This section of the survey asks you to consider the impact of the Women's Program. Please answer Part C only if your organization has ever received funding from the Women's Program. 

  1. Please read each statement below and check the response that best represents your opinion.
  Strongly agree Agree Neutral Disagree Strongly disagree Don't know
a. Women's Program funding has enabled our organization to undertake initiatives it otherwise would not have been able to pursue            
b. Women's Program funding has helped our organization to achieve some of its objectives in advancing women's equality            
c. Women's Program funding has helped our organization to leverage other sources of funding            
  1. Thinking about what your organization has accomplished with Women's Program funding, what has changed as a result of those initiatives? 

    Please do not describe your organization's activities.
    Instead, describe the outcomes of those activities, or what has changed because of them.
    Example of an activity:  Preparing an instruction manual on how to run for political office
    Example of an outcome:  More women running for political office

    ___________________________________________________________
    ___________________________________________________________
    ___________________________________________________________
    ___________________________________________________________
    ___________________________________________________________

  2. Have any of your organization's initiatives had unintended consequences, either positive or negative?
    ___ Yes, positive unintended consequences
    ___ Yes, negative unintended consequences
    ___ No unintended consequences (Go to Part D)
    ___ Don't know (Go to Part D)

  3. Please describe any positive or negative unintended consequences of your initiatives. 
Positive Negative
   
   
   
   
   
   


Part D: Relevance of the Women's Program

The final section of the survey asks you to consider the ongoing relevance of and need for the Women's Program.

  1. Please read each statement below and check the response that best represents your opinion.
  Strongly agree Agree Neutral Disagree Strongly disagree Don't know
a. The Women's Program responds well to new and emerging groups            
b. The Women's Program responds well to new and emerging issues            
c. The Women's Program is still relevant to advancing women's equality in Canada            
  1. What would be the consequence for your organization if the Women's Program did not exist?
    __________________________________________
    __________________________________________
    __________________________________________
    __________________________________________
    __________________________________________
    __________________________________________

    ___ No opinion/don't know

  2. What do you think would be the consequence for the advancement of women's equality in Canada if the Women's Program did not exist?
    __________________________________________
    __________________________________________
    __________________________________________
    __________________________________________
    __________________________________________
    __________________________________________

    ___ No opinion/don't know

  3. In your opinion, how could the Women's Program be improved?
    __________________________________________
    __________________________________________
    __________________________________________
    __________________________________________
    __________________________________________
    __________________________________________
    __________________________________________

    ___ No opinion/don't know

Thank you for taking the time to complete this survey.
Please return it in the enclosed self-addressed, postage-paid envelope to:

PRA Inc.
500 - 363 Broadway
Winnipeg, Manitoba  R3C 3N9
Or you can fax it back to us toll-free at:
1-800-717-5456


Evaluation of the Women's Program
Case Study Interview Guide
  1. To start, please tell me something about your organization.
    • When was it established?
    • What is its mandate?
    • Whom does it represent?
    • What kinds of activities does it engage in?

  2. How often has your organization received funding from the Women's Program (WP)? For what initiative(s)?

  3. We are interested in knowing more about your initiative(s).
    • What need or issue was the initiative designed to address?
    • What were the goals and objectives?
    • How did it seek to achieve its goals and objectives (i.e., what were its strategies and activities)?
    • What partnerships did it involve, either financial or non-financial?

  4. As you know, each initiative funded by the WP is expected to align with one or more objectives of the WP and address one or more of the WP's areas of focus.  Which objective(s) was your initiative designed to address? Which area(s) of focus was your initiative designed to address?

  5. What outcome(s) was/were expected to result from your initiative(s)?

  6. (Ask each of the following sub-questions for each expected outcome)
    • Has your initiative been successful in achieving or making progress toward (expected outcome
    • If yes, what evidence demonstrates its success? Please give examples to demonstrate its success.
    • What factors contributed to this success?

  7. Has your initiative had any unexpected outcomes, either positive or negative?

  8. Does your organization plan to continue working on this issue/in this area?  If yes, please describe any plans you may have for future related initiatives.

  9. In your opinion, is the WP relevant to the advancement of women's equality? Why or why not? What would be the consequence for your organization if the WP did not exist?

  10. What lessons have been learned from the experience of your initiative?

  11. Do you have any other comments?