"Not everybody buys at Wal-Mart," Farid said. "When you are selling a $30 T-shirt, it doesn't break you to spend $1 more to make it here versus overseas."
Even clothing companies firmly entrenched in Southern California said they've been affected by changes in the local garment industry.
Velvet, based outside Los Angeles, contracts out its clothing-making to third-party manufacturers in LA, Chief Financial Officer Michael Rosen said. The company has seven retail stores and also sells its clothing to boutiques and department store chains such as Nordstrom.
Over the years, Velvet has seen its costs rise as more money is required to monitor factories to ensure they are abiding by labor laws, Rosen said.
"There are many fewer production facilities here, so it's harder to find manufacturers that are properly licensed," Rosen said.
"The oversight that is required is much more."
Local garment workers said they're worried about their livelihoods if the city loses more clothing companies.
Marina Neza, 66, was one of the workers who lost her job as part of American Apparel's latest layoffs.
Neza said she used to make about $500 a week as a sewing machine operator. Over 32 years in the industry, Neza said she's been fortunate to work for many apparel makers in Los Angeles that paid decent wages.
But Neza said she's worried about finding another job to make ends meet. Her husband is sick, she said, and can work only part time.
"Apparel manufacturing has been my life," she said. "There are many, many companies that have moved out of here. I'm not sure about finding work."