The Olympic Village for the 2010 Winter Games could be an expensive proposition for Vancouver taxpayers, now that a lender has pulled out.