Denise & Eric Copeland
12218 Big Canoe, Jasper Georgia, 30143
March 5, 2019
[Director Name]]
Big Canoe Property Owners Association
10586 Big Canoe, Jasper, GA 30143
Dear [ ]:
We are writing to express our concern about the recent Facilities Planning Survey designed and administered by the H. Chambers Company. The POA President’s Report, published in the March 2019 edition of Smoke Signals states the following:
[the survey] is being done to help us determine how we can maintain what we have. Neighborhoods deteriorate slowly. Having objective sets of eyes suggesting to us where improvements can be made to maintain our community will help us control maintenance costs and hopefully reduce more significant capital expenditures in the future.
Our principal focus is ensuring we maintain what we have and not let it deteriorate. If, in the course of this evaluation, we find opportunities to add value to our facilities, and do that at reasonable cost, we will consider those ideas.
Whatever plan gets developed will be a community plan—not a board plan.
As Big Canoe property owners concerned about the current trend of out of control spending, this is an approach that we can fully support. However, one cannot achieve these reasonable and sound business goals using information generated by the Facilities Planning Survey designed and administered by the H. Chambers Company. The implementation plan for executing their contract deliverables; the poorly executed sampling plan for distributing the survey; the significant design flaws in the survey; and the scope and content of the questions render it meaningless for providing accurate and relevant information to use as a basis for long range capital improvements and budgeting as described in the Smoke Signals article.
Chamber’s Implementation Plan.
The purpose of the survey and how the results will be used are described on the first page of the survey as follows:
After the survey reveals which projects the community considers the most important, the staff will do a programming study, a conceptual design, site selection, and budget estimates. The projects will then be compared based on value to the community, affordability, immediacy, availability of capital, and dependencies between the projects. The staff, in conjunction with the Board of Directors and the Long Range Planning Committee, will schedule these projects over the next 10 years. This schedule will be integrated with the ongoing maintenance and replacement projects. As each project’s schedule date approaches, the normal project planning, designing and construction processes will be followed, including property owner votes to approve any that are over the adjusted $1M limit.
This implementation plan, which is already being executed, differs significantly from the methodology described in the POA President’s Smoke Signals article. In this version of the story
- The survey results define the “will of the community” and become the demand signal and justification for new capital projects.
- All the community approved projects, as identified by the survey, will go through a programming study, conceptual design, site selection, and budget estimates.
- The projects will be prioritized.
- The staff, Board of Directors and Long-Range Planning Committee, will schedule these projects over the next 10 years.
- As each project’s schedule date approaches, the normal project planning, designing and construction processes will be followed, including property owner votes to approve any that are over the adjusted $1M limit.
In “Chamber’s World,” property owner input ceases when the survey closed out on February 25, 2019. The next opportunity for property owner involvement comes at the point where a property owner vote is required to approve any projects that are over the adjusted $1M limit. Even if property owners receive the occasional “information briefs” referenced by the POA President, there is no guarantee that significant changes can be made at that point, and even if changes are made, a significant level of effort, manpower, and resources would have already been expended.
Poorly executed sampling plan
Chamber’s sampling methodology and survey distribution were significantly flawed in that the target audience was not clearly defined and many property owners did not receive the survey.
- The survey was targeted to “Big Canoe Residents”. The survey overview clearly states that “It is important that every Resident of Big Canoe completes this Survey.” However, they do not define the term resident. Is a minor child of a property owner a resident? How about adult children, in-laws, parents, or other people who reside in Big Canoe with some unspecified frequency? How is a resident defined for a lot with no structure? What about residents of rental property?
- Over the last week or so, we mentioned the survey to other Big Canoe property owners to see if they had the same concerns and issues that we have regarding the survey and process. A number of property owners had no idea what we were talking about; several had technical problems with the survey tool; and others did not know that they could share the survey or who to share the survey with – most Big Canoe survey’s or votes are constrained to one input per lot number.
Given these flaws, there is no way that the survey results can be remotely characterized as representing Big Canoe “Community” priorities.
Survey Design Flaws
The H. Chambers Company specializes in country club, resort, and other Private Club Communities. Big Canoe is not a Private Club Community and, by definition, has many unique characteristics, issues, and constraints that have to be addressed that are significantly different than the normal Chamber’s contract. Big Canoe is private mountain community that has facilities, activities, and amenities to provide owners an extraordinary quality of life and lifestyle. It also has numerous natural areas, community spaces, trails, and other non-resort items that need to be factored into the long-range plan that Chambers does not consider. The Chambers survey focuses on what they know best – the private club industry. The majority of the survey focuses on seven resort-like facilities: This is mostly a repeat of the material that was presented in the focus groups where there was significant push-back from property owners regarding scope and scale of the effort. The survey had one small section on Other Potential Enhancements / New Amenities and another small section to reflect overall priorities. Each section of the survey has significant flaws discussed below but one overarching design flaw stands out and needs to be addressed up front.
The majority of the questions are in the form of a Likert Scale. The traditional Likert Scale represents a possible range of answers with the two extremes on the left and right; the neutral response in the middle; and one response on each side of neutral that falls between the extreme and the neutral response. For example, the possible responses might be (from left to right): Very Important, Important, Neutral, Unimportant, & Very Unimportant. In many cases, people who want to indicate one of the extremes as their answer to the question are used to selecting the left-most or right-most answer.
The Chambers’ survey is fatally flawed from a human factors perspective in that they added an additional neutral response, and placed it in the extreme right position. The Chambers survey had the following scale: Very Important, Important, Neutral, Unimportant, Very Unimportant, & No Opinion. There is no valid statistical reason to include two neutral answers and certainly no reason or justification to replace the right-most response (where most people expect to find the “Very Unimportant” response) with a “No Opinion” response. This flaw basically invalidates or cancels out many of the responses where the respondent indicated, “Very Unimportant.” Intentional or not, the end result is that the survey fatally skewes the output to reflect more positive responses.
Regarding the Section-by-section design flaws:
1. CLUBHOUSE FACILITY USAGE AND POTENTIAL ENHANCEMENTS
The introduction to this section creates a response bias in the survey by placing the questions in the context of two statements that are falsely presented as “facts”:
- “We have a Clubhouse with great traditions that has served us well throughout the years.”
- “We must also prepare the Community for current and future trends in the industry.”
The implication is that we need to keep and maintain the status quo and allocate money and resources for enhancements to make it better.
As best as I can tell, the Clubhouse is one of the only Big Canoe POA facilities that generates a loss year-after-year. I am not sure that most property owners think that this negative revenue generation is one of the “great traditions that has served us well throughout the years.” Also provided, as a fact, is that the community must keep up with the current and future trends in the “private club industry.” Unlike a private club community where everything revolves around the Clubhouse, Big Canoe is a private mountain community that has facilities, activities, and amenities to provide owners an extraordinary quality of life and lifestyle. The Clubhouse is one of many competing facilities in Big Canoe and intrinsically has no more importance or priority than any of the other facilities that compete for owner’s time and resources.
The questions regarding Clubhouse usage are flawed in that the wording injects a response bias towards generating higher “Utilization Rates.” The fixed costs of managing and maintaining the Clubhouse do not change week to week yet the survey only asks about usage “while residing at Big Canoe.” If part time owners do use the Clubhouse frequently when in residence, but are only in residence two weeks a year, that is going to artificially inflate the actual usage seen at the club from week to week.
The next eleven questions deal with the Clubhouse enhancements we evidently need to consider in order to keep up with industry trends. To my knowledge, no one has ever provided a correlation, much less causation, between the current low utilization rate of the Clubhouse (and the corresponding negative cash flow) and the 11 items listed in the survey. Spending additional money to enhance a facility that is already losing money is not a recipe for success unless the enhancement generates sufficient additional income to recoup the cost of the enhancement and add an increase to the Clubhouse bottom line.
2. GOLF COURSE, WELLNESS CENTER, SWIM CLUB, AND RACQUET CENTER
The next several sections (Golf, Wellness Center, Swim Club, and Racquet Center) do not even ask for input if the respondent did not use the facility in the last year. This is a serious design flaw as it generates different sample sizes for the responses to these questions making it impossible to aggregate these sections with the other data and obtain meaningful information. If the questions are answered, the survey has the same issue described above where questions regarding usage inject a response bias towards generating higher “Utilization Rates.”
3. WILDCAT RECREATION AREA USAGE AND POTENTIAL ENHANCEMENTS
This section of the survey has the same issue described above where questions regarding usage inject a response bias towards generating higher “Utilization Rates.” The next six questions deal with the proposed enhancements we evidently need to consider in order to keep up with industry trends. To my knowledge, no one has provided a correlation, much less causation, between the current utilization rate of the Wildcat Recreation Area and the six items listed in the survey. Spending additional money to enhance a facility that may not need any enhancements is not a recipe for success unless the enhancement generates sufficient additional income to recoup the cost of the enhancement and add an increase to the bottom line.
4. MARINA USAGE AND POTENTIAL ENHANCEMENTS
Same flaws discussed in the section above.
5. OTHER POTENTIAL ENHANCEMENTS / NEW AMENITIES
This section is problematic because the description of the “enhancement” contains an eight to ten-word title with no corresponding scope or explanation. The lack of detail in the descriptions makes it impossible to provide an informed response regarding its usefulness and how it might fit into an integrated priority list.
OVERALL PRIORITIZATION
These three sections of questions are problematic because the survey forces a binary choice for the respondent: – it is either good or bad. Without understanding the scope and magnitude of the “enhancement” it is impossible to provide a meaningful response.
Insufficient Scope and Content
Finally, the survey does not elicit the necessary and sufficient data required to conduct the analysis described in the POA Board President in the Smoke Signals article. Even if the results of the survey were not irrevocably skewed due to poor design and execution, the contents are incomplete and insufficient to produce the relevant data necessary conduct the analysis described by the POA President’s Report. It does not take a lot of critical thinking skills to readily see that the survey simply cannot produce the data necessary to provide actional information regarding: maintaining what we have; controlling maintenance costs; and reducing capital expenditures in the future.
The narrow subset of capital improvements that were the basis of the survey failed to address any of the issues and concerns that we feel are important parts of the long-range planning calculus. There were few, if any, items that we were able to respond to with an answer other than “Not Very Important”. While the survey did provide text boxes for write-in items, they are an afterthought. We have no confidence that free-form text input across multiple surveys can be accurately categorized, aggregated, and made part of the results for analysis in any meaningful manner. This completely disenfranchises the property owners that were forced to record their inputs into free form, write-in text.
Conclusions and Recommendations. This survey has significant flaws that render it meaningless for providing accurate and relevant information to use as the basis for an integrated long-range infrastructure plan that balances maintaining what we have; controlling maintenance costs; reducing capital expenditures; and identifying opportunities to add value to our facilities, at a reasonable cost Furthermore, it fails to address a significant portion of the Big Canoe infrastructure including the numerous natural areas, community spaces, trails, and other non-resort items that need to be factored into the long-range plan.
As new homeowners in Big Canoe, we are in total support of the POA position that we, as a community, need to develop a plan that balances maintaining what we have; controlling maintenance costs; reducing capital expenditures; and where possible, identify and program opportunities to add value to our facilities at a reasonable cost. This survey absolutely cannot achieve this objective for the reasons stated. Apologies for the length and detail presented in this letter but we wanted to convey to the board that our concerns are valid, evidenced-based, and not just empty “sound bites.” In addition to pointing out the problem, we also have a solution.
We recommend that the survey be redesigned and be reissued by a community-led group of property owners that have some “skin in the game.” Also, realizing that talk is cheap and resources are expensive, we are willing to volunteer to be part of the process of developing a proper data-collection plan, designing the survey, and devising a sampling plan that can produce results that are truly representative of the community.
Sincerely,
Denise & Eric Copeland
12218 Big Canoe, Jasper Georgia, 30143
EDITOR’S NOTE: I wanted to add that I have received information from multiple sources that the preliminary Chamber’s Report has already been released. It has been reviewed by the Board, Select Committees, and the GM & Staff. It has been sent back to Chamber’s with instructions to CHANGE / REFOCUS much of the report. I am concerned that we have paid $125,000 for a consulting process that involves collecting Property Owner input – but then closed door “instructional guidance” is given to Chambers as to what the formal and final Report needs to say. Obviously this is yet another item to add to the list of flaws in this process, as have been very astutely and professionally pointed out by Denise & Eric, who I think did a great job of presenting this. Don’t we have some great minds in this community!?!?! I just wish I knew why we weren’t using our own resources more. I fully support Denise & Eric’s idea of a community-led group of property owners that have some “skin in the game” to redesign and reissue a survey. I’ve got an article on how we could go about this coming out in weeks. We’ll discuss double-blind registration and data collection, as well as putting together lists of multi-threaded questions that actually matter, and could provide us with a wealth of information!
Be the first to comment