Major averages were coming off a losing week as investors grew worried that the Fed's potential move to taper stimulus could slow down the economic recovery.