Tara Flynn, a rising star at a TV production unit of 21st Century Fox, walked into her boss’s office last August and told him she was quitting and joining streaming-video giant Netflix Inc.

The news was not well-received.

“Netflix is public enemy No. 1,” said Bert Salke, the head of Fox 21 Television Studios, where Ms. Flynn was a vice president, according to a Netflix legal filing.

When Netflix finalized Ms. Flynn’s hire a few weeks later, Fox sued, accusing it of a “brazen campaign” to poach Fox executives. In response, Netflix argued Fox’s contracts are “unlawful and unenforceable.”

The ongoing legal battle is just one sign of the escalating tensions between Netflix and Hollywood as the streaming-video company moves from being an upstart dabbling in original programming to a big-spending entertainment powerhouse that will produce more than 70 shows this year.

It is expanding into new genres such as children’s fare, reality TV and stand-up comedy specials—including a $40 million deal for two shows by Chris Rock. The shift has unnerved some TV networks that had become used to Netflix’s original content being focused on scripted dramas and sitcoms.

Netflix’s spending on original and acquired programming this year is expected to be more than $6 billion, up from $5 billion last year, more than double what Time Warner Inc.’s HBO spends and five times as much as 21st Century Fox’s FX or CBS Corp.’s Showtime. It spent close to $10 million an episode on “The Crown,” a lavish period drama about a young Queen Elizabeth II.

Its shock-and-awe spending—combined with that of Amazon and other new players—is driving up costs industrywide and creating a scarcity of people and equipment.

“You just can’t compete with someone coming in with fresh money, low overhead and a lot less baggage than you,” said Darrell Miller, an entertainment lawyer at Fox Rothschild LLP. One veteran television executive likened Netflix’s onslaught to Genghis Khan’s.

TV stars are demanding “movie star” salaries of some $250,000 per episode when they previously were content with half that, according to studio executives. Competition for “A” team camera crews, sound engineers and postproduction specialists is fierce.

“It’s a feeding frenzy to get the best people,” said Jeff Greenberg, a prominent casting director.

TV networks and studios helped fuel the rise of what is now a competitor. When streaming video led to a plunge in DVD sales about a decade ago, licensing shows to Netflix was seen as a new way to cash in and extend the financial life of expensive programming. With Netflix now seen as a direct rival for original programming, some media companies are cutting back on those licensing deals, including Discovery Communications Inc. and Scripps Networks Interactive.

“We of course have flare-ups because we compete for people, we compete for projects,” said Netflix Chief Content Officer Ted Sarandos. But “we are in this together” with media companies, he said.

About 20 other 21st Century Fox employees not under contract have recently jumped to Netflix, a person familiar with the matter said. Netflix also recently plucked away Stacey Silverman, a senior executive at Comcast Corp.’s NBCUniversal Television, as well as executives from Sony Corp. and Walt Disney Co., among others. (21st Century Fox and Wall Street Journal-owner News Corp share common ownership.)

Faced with spiraling costs, TV chiefs are redoubling efforts to discover new writers, show creators and less-expensive stars. Scripps created an HGTV show called “Home Town” starring a fix-it couple found on Instagram, and it has several in the works with YouTube stars. NBCUniversal said it would make shows with BuzzFeed based on popular web video and news content.

Producers and agents are beginning to talk about the downsides for talent of being on Netflix shows and movies—for example, by saying they don’t get the promotion they deserve on Netflix’s crowded shelf of content—but many Hollywood stars still relish the chance to be in a Netflix original. Netflix tends to offer more money up front along with a more flexible filming schedule due to fewer episodes, and several of its shows have won critical accolades and industry buzz.

Some TV industry executives and Wall Street skeptics question whether the company can add enough subscribers, especially in international markets, to support its breakneck spending pace and justify a $60 billion market capitalization that values Netflix at more than 300-times its 2016 earnings. Netflix’s overall subscriptions grew 25% in 2016 from the previous year to nearly 94 million.

MoffettNathanson analyst Michael Nathanson estimates Netflix this year will have negative free cash flow—a measure of profitability—of about $2 billion due to elevated spending on original shows, which require a higher upfront outlay than library deals.

Netflix has financed the spending by borrowing money, increasing its debt burden more than 17-fold since 2012 from $195.8 million to $3.4 billion.

Investors and Netflix executives are focused on subscriber growth over any other metric and have been encouraged by a surge in international customer additions over the past couple of quarters. Last quarter, it added 5.12 million subscribers abroad, beating expectations. Shares are up 44% in the past year.

Netflix’s desire for domination is on display in its push into comedy. It has gone after big name stand-up comics that used to call HBO home, including Mr. Rock, Louis C.K. and Amy Schumer, with an aggressiveness not seen since CBS’s famed talent raids on NBC in the late 1940s, when it wooed away radio stars such as Jack Benny and Edgar Bergen. Mr. Rock’s $40 million payday is more than double what he was getting at HBO, according to people familiar with the terms.

Mr. Sarandos, the content officer for Netflix, declined to comment on Mr. Rock’s deal.

Netflix is also attracting big-name reality show creators like David Broome of the hit NBC weight-loss show “The Biggest Loser.” In an interview, Mr. Broome said he struck a deal with Netflix that will give it first dibs on all his future creations ahead of other networks, and allow him to sell merchandise world-wide based on his Netflix shows, with the streaming service receiving a cut.

Mr. Broome said he made the decision after an NBC show he was working on last year—“Strong”—failed to gain traction because of what he said was awkward scheduling, including a clash with basketball star Kobe Bryant’s final NBA game. He said he realized Netflix would unchain him from being “jammed into a hole” in traditional TV.

He has since created for Netflix “Ultimate Beastmaster,” an international competition series that launched in February featuring an elaborate obstacle course. The high budget he was given allowed him to make the genre, which many consider to be lowbrow, “feel like a piece of film,” Mr. Broome said. He did a shoot with more than 50 ultrahigh definition cameras, which he said “never in the history of our industry has been done.”

Some shows with high expectations haven’t fared well on Netflix. Chelsea Handler’s late-night-style talk show isn’t generating buzz, and the Q Scores Company, a research firm that measures appeal and popularity, said her scores have fallen since she made the move from TV to streaming. The show’s original three episodes per week have been cut to one.

Mr. Sarandos said Ms. Handler is a “very funny and provocative host” but the show is “a work in progress” and “still getting there.”

A spokesperson for Ms. Handler didn’t respond to requests for comment.

Netflix must keep things cordial with Hollywood’s traditional studios to make deals for reruns and movies, and prevent them from turning to its main competitors, Amazon and Hulu. The vast majority of viewing on Netflix is still of the licensed material, its executives say. Measurement firm Luth estimates more than 70% of Netflix viewing is for that content.

Food Network-owner Scripps grew wary when Netflix began making original cooking and lifestyle programming, with another 20 unscripted TV shows under development. Late last year, Scripps decided not to renew a rerun-licensing deal with Netflix, which was turning into a direct competitor. “That was sort of untenable,” said Scripps Chief Operating Officer Burton Jablin.

Mr. Sarandos said “recycled content” from companies such as Scripps has come and gone from Netflix’s service, but viewing hours haven’t dropped as a result.

Netflix set off tensions in the film world when it offered $120 million to $130 million for world-wide rights for “The Irishman,” director Martin Scorsese’s coming movie about the death of Jimmy Hoffa, which will star Robert De Niro, Al Pacino and Joe Pesci. Viacom Inc.’s Paramount Pictures previously had a deal to distribute the movie domestically for $15 million and independent company STX Entertainment had bought the foreign rights for $50 million, said people with knowledge of the arrangements.

As the movie’s budget ballooned due to costly visual effects, the movie’s backers turned to Netflix, which would stream it online and release it simultaneously in a handful of theaters. While Paramount was willing to let the film go, STX insisted it still had foreign rights and Netflix’s bid was invalid, leaving tempers high and the issue unresolved, the people said.

“We’re highly interested in Martin Scorsese making the film for Netflix,” Mr. Sarandos said.

In other cases, Netflix has earned praise for investing in film categories that major studios have largely abandoned—such as mid- and low-budget specialty movies and star vehicles, such as Will Smith’s coming “Bright,” instead of franchise movies and blockbuster sequels.

In television, star producers and actors can make huge sums of money for years when reruns get sold, but Netflix demands world-wide rights to shows for long periods of time, potentially curtailing income.

“When you make a deal with Netflix, you’re making a deal with the devil financially because, basically, they are buying out all rights,” said Lee Dinstman, a partner at the APA talent agency who represents several show creators. He recently chose a Hulu offer over one Netflix made to his client due to potentially bigger international sales. He said every show’s prospects are different and that Netflix at times may offer the best route.

Sony Television executives felt slighted by the way Netflix abruptly canceled the dark drama “Bloodline” after deciding that it wasn’t achieving the global viewership to justify its high price tag north of $7 million an episode, people familiar with the situation said. Sony can’t shop the show to other buyers for at least a decade, since Netflix has the global rights.

People close to Netflix said it paid for those potential future profits up front, arguing Sony didn’t lose money. Sony hasn’t approached Netflix to buy those rights back, the people said.

A Sony spokeswoman declined to comment.

As Netflix’s shelf of original shows has become crowded, it has cut back on individual shows’ marketing budgets, some producers and agents complain. That consideration is becoming “top of the list in the negative column against Netflix,” a top talent agent said. “They are not promoting the vast majority of the shows they make. It’s starting to be felt by everybody.”

Mr. Sarandos said the company’s approach of using its data to market shows in the app to people who are most likely to watch is more cost-effective than big TV or billboard ad campaigns.

He added a lot of Netflix originals aren’t meant to be everyone’s favorite show. “Doing Coca-Cola level of marketing for ‘Master of None’ doesn’t make a lot of sense,” Mr. Sarandos said, referring to a comedy starring Aziz Ansari, targeted at younger urban audiences.

“Culture is not getting made” because of Netflix, said Propagate Content’s co-CEO Ben Silverman, who helped create several hit network shows like “Jane the Virgin” and a now-canceled, big-budget Netflix drama “Marco Polo.” Because you are only reaching Netflix’s subscribers, “you’re missing so many people,” said Mr. Silverman.

Two of television’s hit shows, NBC’s “This is Us” and AMC’s “The Walking Dead,” average 15.2 million viewers and 16.4 million viewers, respectively, according to Nielsen. Netflix doesn’t release viewership numbers, so creators have only a limited idea of how their shows perform.

Mr. Sarandos disputes the idea that Netflix shows aren’t entering the cultural conversation, pointing to critically acclaimed series like “Stranger Things” and “Making a Murderer.”

NBCUniversal is touting to Hollywood its ability to promote a show across its entire suite of cable networks, including USA, Oxygen, Syfy, E! and Bravo, as well as other businesses like theme parks.

Alternatively, Scripps and Discovery executives are selling producers and talent on the fact that their companies’ channels only showcase a few themes—like home, food and travel for Scripps—so creators are guaranteed a built-in fan base.

Some TV executives are trying to turn one of Netflix’s supposed selling points—releasing entire seasons of shows at once for users to binge—into an argument against the company.

“I think a lot of people have now come around to say it’s nice when your show is out there 10 or 13 or, in the case of ‘The Walking Dead,’ 16 weeks. That is cultural currency,” said Joel Stillerman, AMC’s president of programming. Advocates say the “water-cooler effect”—friends chatting about and spreading excitement for a show—can help build its momentum.

Netflix has said its consumption research shows subscribers want to watch shows at their own pace rather than be tied to a network’s schedule.

Many show creators say Netflix executives generally don’t pass down heavy-handed “notes” to dictate a show’s direction like traditional TV outlets do.

“If you disagree with a note from Netflix or Amazon, you can have a creative discussion about it. It’s not viewed as, ‘You didn’t take my notes? You’re an enemy and I’m going to cancel your show,’ ” said “Gilmore Girls” creator Amy Sherman-Palladino.

After a “wonderful” experience creating the last season of “Gilmore Girls” for Netflix, Ms. Sherman-Palladino said: “My network days are probably done.”

—Ben Fritz contributed to this article.

Write to Joe Flint at joe.flint@wsj.com and Shalini Ramachandran at shalini.ramachandran@wsj.com