Tuesday, September 23, 2008

Survey results are in

In August, SF K Files visitor Abigail Marks posted a link here to a survey asking parents about their experiences in the assignment process this year and for their ideas of what might need to be changed. Marks finally has the results, and wanted to share them with you! Her write-up of the results turned out to be quite long. For the full results, please see http://kassignmentsurvey.blogspot.com, or email Marks at iamabby@comcast.net and she'll send you a Word document.

Here's a brief summary of the results:

The tentative findings suggest that most families would support changing the neighborhood assignment system to one that incorporated some kind of zone or geographic area where a family would have preference, with an ability to enter a lottery for schools outside this area. The key element in such a system would be to safeguard parents’ abilities to get an assignment of their choice.

Opinion about whether people should be given some kind of preference for a school nearby is split. A sizable number want this, and a somewhat smaller but still sizable number don’t. When we divided the sample into three groups of zip codes, the western zip codes had a much higher percentage in favor of neighborhood preference than did the eastern zip codes (76% vs. 36%). However, both eastern and western zip code groups preferred the mixed system (neighborhood component plus city-wide lottery).

My suspicion is that with a more demographically balanced survey the zip-code discrepancy might not be so large, but the significance of this difference suggests that: among parents active on the internet, those who live in neighborhoods where schools are perceived to be higher quality are more likely to want neighborhood preference, where those who live in neighborhoods where there are fewer schools perceived to be higher quality wish to preserve their right to choose city-wide.

However, there are indications that the issue of neighborhood preference is complicated and charged. When respondents were first asked about neighborhood preference, a full 34% stated that they did not want it, and wanted to choose freely from all San Francisco schools. However, when asked to rank their choices of an assignment system, only 13% chose a city-wide system with no provision for neighborhood or geographical location. What can this result mean? Perhaps that the aspect of preserving choice is key. It may be that, if the choice still remains to be able to attend any school in San Francisco, a system which combines neighborhood and other factors, and which allows city-wide choice may be more attractive.

The more serious finding of this survey, however, is the lack of trust among parents for the way the District/EPC runs the assignment process.

The image of the District and the EPC that emerges from the survey is one where the District is not procuring or is actively withholding information (about available seats) or deliberately concealing or misleading parents (about weight of first choice, or covering up assignment errors). There is a lack of trust that the computer system will not make errors, that the coding is not being sufficiently checked so that assignment errors will occur, and –perhaps most seriously – a feeling that when errors do occur, that the District will do nothing to reveal this fact, or to take any action until forced. When trying to remediate a problem or error, there is a sense that the District will act in an unpredictable way, without taking community input into account.

This lack of trust may also be exacerbated by the high number of first time kindergarten families who got none of their Round I choices this year (around 45%). This has implications when designing a new assignment process –as parents may not trust that any new system will have room for their needs and not lower their chances of getting a school that is a good fit for their family.

Suggestions of how to make use of these tentative findings:

1. Solve the easy-to-fix problems quickly: in the lottery for the 2008 year, run the siblings first. Let people know the spaces available in each school before Round II and subsequent runs. If possible, come up with a system that lets families be on more than one (say, 3?) waitpools in the 10 day count.

2. Strive for more accountability: double-check coding (especially, but not only, in the areas where there were error problems this year). When there is a problem (whether it is a mistake in coding, or a failure to anticipate an extra 300 kindergarten applications than planed for), reveal it quickly and go about fixing it (with community input) right away. Keep promises made to schools and community groups.

3. Strive for more transparency: In addition to the easy-to-fix measures above, make sure District and EPC representatives are giving accurate and consistent information (about weighting first choice schools in Round I, about sibling preference for twins, about whether the diversity index is used in the waitpool runs, about whether students are being tested for language proficiency etc. etc.). Then release information to the public that will prove that the District is being entirely open and forthcoming (computer formulas for assignment, internal written policy on placement issues etc.).

4. Strive for ways of maximizing families’ effective choice. Come up with an assignment system that lets families who want a choice between schools close to home do that, and those who want to have a city-wide choice do that. And take steps to maximize each family’s chance of getting one of the schools they most want. The emphasis on closing the achievement gap and improving schools in all neighborhoods should help.

5. Strive for more community input: in the process of coming up with a new assignment system, or indeed with any issue, cast your net wide. Solicit ideas from parents of present and future students, from community-based organizations like Parents for Public Schools and the SFAME. Run your top ideas past people to see what the unanticipated impact might be, what the holes in the ideas might be. Solicit community input for making those ideas better. Put out more surveys (hopefully a bit more well-designed and thorough than this one, and with more outreach to all SF families)! Work with community and parent groups to implement recommendations, such as those gained from the thorough SERR report.

21 comments:

  1. I wonder what the impact would be if families could list more choices -- perhaps a lot more.

    ReplyDelete
  2. I really do appreciate the effort behind this. And the results are very interesting as showing what a slice of the SF population feels. But this just doesn't have any real meaning to me:

    "Come up with an assignment system that lets families who want a choice between schools close to home do that, and those who want to have a city-wide choice do that. And take steps to maximize each family’s chance of getting one of the schools they most want."

    It's just not possible to make neighborhood matter more without restricting the choices of people who come from outside the neighborhood. There are only a certain number of slots at each school, and if more neighborhood kids get in than fewer kids from outside the neighborhood get in. So saying you want neighborhood AND choice doesn't say that much -- the question is, which matters more and should one be given more emphasis at the expense of the other?

    ReplyDelete
  3. These aren't tentative findings - they're statistically invalid findings based on a survey whose questions - and whose reach - were slanted toward a particular viewpoint. And the response rate is so low that no statistically significant findings could be drawn even if the survey adequately reached San Francisco and assigned appropriate race to demographic factors (race, home language, income, zip code).

    At best, this survey could point to the need of doing some actual survey research around these questions. But it is a stretch to draw any findings from this survey - and frankly, to me it speaks heavily of at least class privilege to do so.

    ReplyDelete
  4. I'm confused by this: "deliberately concealing or misleading parents (about weight of first choice"

    I didn't think the district weighted any of the choices more heavily than another?. When we applied a few years ago I did not get my 1st choice - didn't actually get any of my choices -, but I know LOTS of people who got my first choice school as their 3rd or 4th choice. I just assumed they got lucky. I would imagine though as a school gets more popular and more people list it first, it might begin to appear as though your first choice is waited more heavily?

    ReplyDelete
  5. While I can appreciate the amount of effort that went in to this project I find it disturbing that anyone would try to draw any type of conclusions based on these extremely skewed findings. I suppose theres an inklng here of what upper middle class parents looking for a kindy would like to see happen but was that the survey's intent ? FYI our South of Market zip code wasn't even included.

    ReplyDelete
  6. 9:08am: what is your zip code? did you fill out the survey and the zip code wasn't included in the results? If no one responded to the survey in a particular zip code, it wasn't included in the table.

    ReplyDelete
  7. I didn't think the district weighted any of the choices more heavily than another?

    this is true.

    ReplyDelete
  8. According to the enrollment literature and PPS, the lottery gives priority to your preference list of schools in the following way: when your kid's list of preferences is run, if she/he gets a positive assignment to more than one school, then the assignment offered is the one highest on the list.

    This is a good thing! When we went through the lottery back in the early 2000's, there was no preference given to one's top choice. The assignment was made to the school to which one's child contributed the most diversity. Needless to say, this discouraged parents from trying to fill out the list of (at that time) five schools with anything but the most popular schools. PPS spearheaded the change. So now you can hope to win the lottery against the odds by putting Clarendon or Rooftop at the top, and put a better-odds but still acceptable school further down the list. If your kid's name comes up for both Clarendon and the other acceptable school, you'll get Clarendon.

    I know, people always have stories about people who got Clarendon at spot #4 or #5 or whatever. It was lottery luck. At the exact moment their kid's name came up, there was a spot that matched at Clarendon but not further up that family's list at Grattan or whatever. The fact that you had it at #1 and they had it at #4 is immaterial.

    That is not to say there haven't been coding glitches, notably with McKinley last year, and with various immersion programs including Marshall and of course Alvarado and Flynn. But I do not believe they had to do with people's #1 choices. That part of the system runs well. (Except that the odds of getting Clarendon when 1,000 people put it down too, are, well, teensy. This would be true even if there were no diversity index. 1,000 requests for 25 or so spots just ain't good odds.)

    ReplyDelete
  9. 9:26pm.

    That is a good point. It is true that it won't be possible to give everyone what they want.

    Does anyone have any ideas about how to incorporate some kind of neighborhood preference into the assignment system, while still maintaining an adequate amount of seats so that a city-wide lottery component isn't meaningless?

    If it's not obvious already, I feel pretty strongly that letting the district come up with a new assignment system without parent and community input means risking the development of a system that satisfies no one. My idea of how to combat that is to get as many creative ideas out there about the best system possible (and to get feedback about problems with proposed systems before they are put into place).

    I am a bit surprised (but not too surprised) about the attacks on the survey itself. Yes, the respondents weren't representative of the SF population, but 169 is actually quite a large number of responses for a questionnaire (at least it was in my grad school days), and my guess is that they might be somewhat representative of some part of the population of parents in SF (maybe even of the viewpoints of visitors to this blog?).

    What is the viewpoint that you're thinking the survey was slanted towards?

    Truly, the effect of reading all of these comments is to make one want to do nothing at all, just let things take their course, for fear of being attacked for doing the wrong or inadequate thing. But can't imagine that's what anyone wants, or do they?

    Any other suggestions on the best way of coming up with a better assignment system than the one we have? If you were on a independent task force of parents advising the school board, what would you suggest about the best way to proceed with the development of a new system?

    ReplyDelete
  10. In order to offer suggestions about neighborhood preference I'd need to understand how the current system works. It's never been clear to me. Is the lottery run for the neighborhood kids first, until that group of kids is as diversified as possible and then the system opens up to the rest of the city. Does the lottery pull names from the neighborhood pool until it has an equal amount of kids with no preschool, extreme poverty and/or english is not the primary language at home, and kids without those factors, and then begin running the lottery without regard to neighborhood, continuing to achieve a balance? Also, I understand the goal of the diversity index is to achieve a balance, but it seems like it does not happen at a lot of schools. Is that because the total pool of applicants is much more heavily from one group or another? In other words, if 300 people request Harvey Milk for example, and only 15 of the applicants fall within the no preschool, ESL, or extreme poverty, then a K class of 60 kids would end up imbalanced, correct? It seems this is one of the trade offs of "choice".

    ReplyDelete
  11. if 300 people request Harvey Milk for example, and only 15 of the applicants fall within the no preschool, ESL, or extreme poverty, then a K class of 60 kids would end up imbalanced, correct? It seems this is one of the trade offs of "choice".

    This is exactly correct. This is what has happened. Truly balanced schools will happen only with a return to forced busing (not saying I support that, just stating the fact). Introduce parental preference, get more imbalance. Return to neighborhood schools, get schools that are as segregated as our neighborhoods (very).

    In terms of neigbhorhood preference, it is my understanding that the computer algorithm gives preference to those in the neighborhood who add balance, only turning to those outside the assignment zone when the collection of remaining neighborhood candidates do not provide that preference. So there actually is neighborhood preference in the algorithm already, but it is balanced against achieving more diversity in terms of poverty versus not-poverty (NOT race).

    A couple of caveats though: these zones are not entirely the local neighborhood, but are assignment zones that often encompass streets outside the immediate neighborhood. Visitacion Valley for Alvarado, for example. Also, families who do not have an assigned school, because their school has been closed or converted to a charter, are given a funny status in that the first non-alternative school on their list of seven is considered their neighborhood school for the purposes of the computer algorithm mentioned above.

    ReplyDelete
  12. At the exact moment their kid's name came up, there was a spot that matched at Clarendon but not further up that family's list at Grattan or whatever.

    that's not how it works. each school runs its own lottery. they don't call a name and see where that person best fits; they calculate what profile they need to maintain diversity and go in and pull out the first kid that matches it. they recalculate and repeat.

    ReplyDelete
  13. I'm not sure we can't reconcile the "pure choice" desired by some with the "neighborhood preference" preferred by others with the desire to create a reasonable degree of opportunity for disadvantaged students.

    If we really want to be a "choice" district, the order in which parents or caregivers rank a school should matter, shouldn't it? If 200 parents put a school as their #1 choice, those 200 people should be put in the drawing for that school and the ones chosen will be offered available seats subject to sibling and special need requirements. (Parents who want sibling or special need spaces at a school could be required to list it as #1 or waive the preference. A well-designed enrollment form should be able to clearly disclose this.) If, to create greater equity of opportunity, you want to split the drawing into two drawings with proportional assignment according to "advantaged" and "disadvantaged," fine. (E.g., if 60% of the students in the city are "disadvantaged," 60% of the names chosen will be out of the "disadvantaged" pot and 40% will be chosen out of the other pot.) If there are not enough "disadvantaged" kids put into the pot by their parents or caregivers to fill the spaces available to them, well, you can only do so much outreach; choose kids out of the "advantaged" pot to fill the remaining spaces. After you run all the #1 rank lotteries, run the #2 rank lotteries (excluding people who got their #1 choice) and on down the line. Let people list as many schools as they want in order of preference. At the end of Round 1 DO NOT assign anyone to a school they did not list. AFTER round 1 enrollment deadlines have passed, announce what's available, have parents submit their new ranked lists, and go into round 2. 2A. Follow the same method, except anyone who did not enroll in a school they listed and in which they were offered a space in Round 1 (unless they have some weird hardship that they explain such as they were in a coma) should be run later in Round 2B. For 2B people, they would not be run for any of their ranked schools that filled up in Round 2A. If after Rounds 2A and 2B a person still has received no school they requested, assign them to the school closest to their home with openings. Then open the wait lists. Let people wait list as many schools as they want, give them the first space that opens at one of their wait list schools, and make them choose between their assigned school and their wait list school within one business day. If they don't enroll in an offered wait list school within one business day, they waive their wait list school and accept their assigned school. This allows people who want a neighborhood school to give themselves better odds by prioritizing that factor in their rankings, it allows people who want city-wide choice to give themselves better odds by prioritizing their favorite schools in their rankings, and it allows disadvantaged kids a proportional number of openings in the top schools (which they would have to take responsibility for requesting). It won't make everyone happy--in fact it's not that different from the current system other than allowing people to make more choices. People will still lie to try to game the system. But at least people would understand how it works. OK, start shooting holes:).

    ReplyDelete
  14. At the exact moment their kid's name came up, there was a spot that matched at Clarendon but not further up that family's list at Grattan or whatever.

    that's not how it works. each school runs its own lottery. they don't call a name and see where that person best fits; they calculate what profile they need to maintain diversity and go in and pull out the first kid that matches it. they recalculate and repeat.

    Good point. But the original poster's point is still true, that the kid gets a match with multiple schools, the kid is offered school that is highest on his/her preference list. So if the kid is found to be a match with Clarendon and Grattan, and Clarendon is listed #1 and Grattan #4, then Clarendon is offered.

    The poster who wrote about neigbhorhood preference is also correct. The computer looks first at kids in the assignment zone, and then outside it. So we already have neighborhood preference. Not sure how much more we could have without completely excluding (de facto) from citywide opportunities.

    ReplyDelete
  15. "Truly balanced schools will happen only with a return to forced busing (not saying I support that, just stating the fact)."

    I don't believe this is the case. The school district can't require anyone to enroll in any school. Forced busing would result in "white flight" just as it did in the 70s (or perhaps its modern equivalent--"middle-class flight").

    ReplyDelete
  16. The system only looks at children in the assignment area only if they add diversity to the school. Because siblings are run first, this essentially means that --if most of the families in your school do not have diversity index criteria -- unless you meet one or more of the diversity criteria (poverty, language other than english at home), you are given no preference for your attendance area school at all.

    Also, for many people, their attendance area school is not the closest one (which may be a school with no attendance area), which makes it all the more confusing...

    ReplyDelete
  17. " If no one responded in a particular zip code it was not included in the survey"
    Okey doky, that right there basically skews your findings prejudicially in favor of people who like yourself are more likely to fill in a survey, and those people are, drum roll please, college educated upper middle class parents. That and a preponderance of your respondents were from wealthier parts of the city. I don't know how you did surveys in grad school but when I did this type of research professionally getting representative samples of all thevarious ecomomic/ethnic/geographic etc. groups was considered basic form. You are putting this out there like it means something when really all it means is 167 people of a certain type from certain neigborhoods feel this way. If you begin with a premise you can easily find "data" to prove it.

    ReplyDelete
  18. I thought Abigail was pretty up front about the fact that the survey isn't representative of every part of the city. That doesn't make it meaningless. She is trying to generate ideas to improve enrollment procedures. It's a springboard for discussion.

    Another report that truly is representative of the entire city is the Student Enrollment and Retention Report. I think there is a link on the PPS website, among other places. It has a lot of overlap with Abigail's survey, and comes up with many of the same findings. Abigail's was more "nuts and bolts" and the SERR was more philosophical (using forced choices between diversity and location for example.)

    Thanks Abigail!

    ReplyDelete
  19. If a survey that addresses a topic effecting every part of the city that isn't representative of every part of the city it's not so much a survey as cocktail party chatter.
    Now that we know how 167 predominatly upper middle class well educated parents feel we can draw
    what conclusions ?

    ReplyDelete
  20. For the many of us who have kids in the public schools, who have been active PPS Parent Ambassadors, and who 'took a chance' on a school that wasn't on the radar of our peers, this study certainly provides no new information. For parents like me, it is what I've been living and working against in helping to promote and improve my own school --- and even more importantly --- helping to work towards and ensure quality in all schools and for all kids.

    I realize if you are new to thinking about enrollment, it seems like news. But it ain't.

    It is important to recognize that this is an extremely skewed sample, and does not reflect even the majority of San Francisco families.

    This is not to say that these viewpoints are invalid, but it has to be balanced. I did think it was interesting the west side/east side viewpoint on neigbhorhood schools. But again, among my friends, I knew this.

    As recently as 2-3 years ago, I don't think moving towards an emphasis on neighborhood schools would have worked for elementary. However, I think that now, the idea could fly as more families see a wider variety of schools as options.

    My caveat is: now those on this list see Paul Revere, Ortega, Sunnyside as viable options, but a year ago virtually none on this list did and a whole new crop of parents coming in won't either --- unitl later in the process. Similarly, folks don't realize that even very recently, Alvarado, Miraloma and Grattan wouldn't have been considered top school picks, and now they are seen as great choices.

    However, any movement to neighborhood school really has to ENSURE (and I don't know how this can/will be done) that schools that there are some attractive NEW options in places like Bayview, Mission (i.e. east side schools.)

    Is it closing Malcom X and putting in KIPP? Currently, K-12 kids that live by Malcom X are going to 75 different schools (yes, really.) Clearly, they are not choosing their neighborhood school.

    A lot of families I know are making an active choice to get their kids out of their neighborhood due to things completely out of their control (gunfire, lockdowns, etc.) Moving westside kids eastside clearly doesn't help. But just forcing those kids to stay there and limiting their options is just plan wrong.

    I wish Abby's study would do more to note the realities that these kids face and how the system doesnt' just screw them over.

    We need serious magnets in these neighborhoods. This has in effect happened organically at schools like McKinley, Miraloma, Flynn, Ortega, etc., (all somewhat in the middle of the city - neither east nor west). But is has not spread into more heavily distressed and impovershed parts of our city.

    There really is no simple solution.

    ReplyDelete
  21. A lot of the schools we're interested in are *not* in our neighborhood, so we are not that crazy about the idea of neighborhood preference.

    English-speaking slots in dual immersion programs should not be subject to any neighborhood preference, and yet, I hear they are.

    ReplyDelete