Ray Bolger, the loose-limbed song-and-dance man who became known to millions as the Scarecrow in ''The Wizard of Oz,'' died yesterday of cancer in Los Angeles. He had his 83d birthday last Saturday and lived in Beverly Hills.

Among his many roles on stage, screen and television in a career than spanned six decades, none captured the public imagination more than his appearance in the 1939 movie starring Judy Garland that sent him, along with the Cowardly Lion (Bert Lahr) and the Tin Woodman (Jack Haley), on a journey along the Yellow Brick Road with Dorothy, the girl from Kansas uprooted by a cyclone, in her search for the Wizard (Frank Morgan).

The last survivor of the four, Mr. Bolger also outlived their nemesis, Margaret Hamilton, the Wicked Witch of the West, who died in 1985. The film, in which the Scarecrow's lean, seemingly straw-filled body is propelled by long legs that bend with the wind, is a perennial favorite on television, being shown worldwide at least once a year.

The Broadway stage was Mr. Bolger's first and abiding love. Born Raymond Wallace Bolger in Boston on Jan. 10, 1904, he began acting in amateur theatricals and at one point was dismissed by an insurance company after being caught dancing in a hallway. He Started at 19

His got his first paid acting job at the age of 19 with a repertory company and soon was appearing in vaudeville with Gus Edwards. There followed Broadway appearances in ''George White's Scandals'' (1931), ''Life Begins at 8:40'' (1934) and, in 1936, the musical ''On Your Toes,'' in which he won acclaim for his dancing in the number ''Slaughter on 10th Avenue,'' choreographed by George Balanchine.

Soon he was in Hollywood appearing in such musicals as ''The Great Ziegfeld,'' ''Rosalie'' and ''Sweethearts'' as well as ''The Wizard of Oz.'' But he was inevitably drawn back to the Broadway stage and in the years after World War II he was a regular in the Little Bar at Sardi's, where show-business luminaries gathered.

In 1948 he opened in the musical ''Where's Charley,'' a vehicle that made him as celebrated on the stage as he had already become in film. In the remake of the hoary stage play ''Charley's Aunt,'' he created his most memorable singing number, ''Once in Love With Amy.'' Long after the three-year run of ''Where's Charley,'' he was called upon to sing the lilting ballad and to perform the soft-shoe-dance routine accompanying it almost every time he appeared on television. A Comedian First and Last

Mr. Bolger, who was 5 feet 10 1/2 inches tall, was so thin that in his solo performances in the spotlight he appeared to be much taller. His legs were so flexible he appeared to be disjointed - even disembodied - as he leapt into the air to punctuate a song-and-dance number. Indeed, many who watched him in later years were unable to shake the image of the straw-stuffed Scarecrow flopping about on boneless legs as he lurched down the Yellow Brick Road.

Although he won his greatest acclaim as a dancer, Mr. Bolger considered himself first of all a comedian. For that reason, he was parcticularly gratified by the success of the farce, ''Charley's Aunt,'' the starring vehicle that was produced by his wife, Gwen Rickard, whom he had met in 1924 and married five years later. She remained a strong force in his career until it ended less than three years ago with an injury that removed him from the performing stage.

In mid-1984 he suffered an injury to his right hip that necessitated his receiving an artificual joint. ''I stepped down from the stage and there was nothing there,'' he said afterward. ''I tried to do another show, but I was not up to par and I had to cancel.'' A 'Dancer in Self-Defense'

His last performance was in 1985 as a narrator in ''That's Dancing,'' an anthology that included a Scarecrow dance that had been cut from the final version of ''The Wizard of Oz.''

Early in his career he decided that comedy was his metier and that his gift for dancing was mainly a vehicle to enable him to win the comic roles he sought.

''I was hired as a comedian in my first show,'' he said many years later. ''I became a dancer in self-defense. I was doing a comedy monologue and didn't know how else to get off, so I danced off. I've been dancing ever since, but I'm still a comedian.''

Over the years he danced and sang his way into many shows, including ''Keep Off the Grass'' (1940), ''By Jupiter'' (1942) and ''Three to Make Ready''(1946) During World War II he organized U.S.O. shows that toured American military and naval bases around the world.

Among his films were ''The Harvey Girls'' (1945), ''April in Paris'' (1952) and ''Babes in Toyland'' (1961).

He became a familiar figure on television, beginning in 1952 with his debut on ''The Comedy Hour,'' followed in 1953 by ''The Ray Bolger Show,'' which appeared weekly on ABC. In 1956 he did the NBC-TV series ''Washington Square.''

In 1979, departing from his light-hearted appearances, he made a dramatic appearances as a somber dignitary of the Roman Catholic Church in ''The Runner Stumbles.'' Frequent Returns to the Stage

With all his work in film and television, he continually returned to the stage, appearing in one-man shows and song-and-dance concerts.

In a sense, these concerts were a return to his first attempts in show business. As a teen-ager in Boston, where he attended public schools, he danced on street corners. After brief jobs as an insurance agent and a vacuum-cleaner salesman, he got a job dusting the studio of a noted ballet instructor of the time, Senia Rusakoff. But he was not drawn to classical dance.

He learned the acticg craft with a repertory company headed by Bob Ott, to whom he later gave credit for schooling him in the comedy. Soon his long, rubbery legs propelled him into a style of dance that he was later to make his own, characterized by the swooping steps that seemed to carry him across a stage in one or two bounds.

Leaving the Ott troupe, he went into vaudeville on his own, playing small towns in New England and the mid-Atlantic states before he finally arrived in New York, where he landed a job dancing at movie houses in the Paramount chain between films. It was there that Gus Edwards found him in 1926 and put him on the Broadway stage.

Among the awards Mr. Bolger received were the Tony award in the 1948-49 season and two Donaldson awards for best performance. In 1980 he was elected to the Theater Hall of Fame.

Surviving is Mr. Bolger's wife, the former Gwen Rickard. The couple had no children.

Mr. Bolger died at the Nazareth Nursing Home in Los Angeles. A funeral service will be held on Monday at the Good Shepherd Church in Beverly Hills.

Raymond Bolger