Japan Invests in Developing Its Own AI Chatbots To Revolutionize Japanese Language Processing

Japan is investing heavily in developing its own AI chatbots, similar to ChatGPT by OpenAI. Government and tech giants like NEC, Fujitsu, and SoftBank are dedicating substantial funds to create Japanese-language artificial intelligence (AI) systems based on large language models (LLMs).

Existing AI Chatbots Might Not Grasp the Intricacies of Japanese Language and Culture

ChatGPT and other LLMs rely on vast datasets from publicly available sources to learn natural language patterns and predict text based on context. While ChatGPT's remarkable conversational abilities have intrigued and alarmed researchers, there's a particular challenge in Japan.

AI systems trained on non-Japanese languages often struggle to comprehend the nuances of the Japanese language and culture. Japanese sentence structures diverge significantly from English, necessitating a translation process for queries and responses.

Moreover, Japanese writing encompasses 48 basic characters and over 2,000 Chinese characters (kanji), often with multiple pronunciations, making the language highly intricate. In Japanese, ChatGPT occasionally generates exceptionally rare characters and unfamiliar words, explains Keisuke Sakaguchi, a Tohoku University researcher in natural language processing.

To be valuable and commercially viable, experts noted that LLMs must accurately reflect not only language but also cultural practices. Researchers created Rakuda, a ranking system to assess LLMs' sensitivity to Japanese culture by comparing their responses to standard prompts.

GPT-4 currently outperforms Japanese LLMs in cultural appropriateness and fluidity. A Japanese LLM project, utilizing the supercomputer Fugaku, backed by multiple institutions, aims to release an open-source LLM next year with at least 30 billion parameters.

Japan's Ministry of Education is funding another LLM tuned for scientific research, with a release expected in 2031, potentially costing ¥30 billion (US$204 million).

ChatGPT's Upcoming Office Expansion in Japan

OpenAI, the creator of ChatGPT, announced earlier this year their plans to open an office in Japan and expand its presence in the country. CEO Sam Altman expressed the company's commitment to engaging with Japanese talent and enhancing its services for the local market.

While specific details about the office's opening were not provided, Altman mentioned that more information would be available in the coming months. Japan is a significant market for ChatGPT, with over a million daily users in the country.

During a meeting with Prime Minister Fumio Kishida, Altman discussed both the advantages and risks associated with AI technology. The Japanese government is considering the use of AI in government ministries and agencies, with a focus on addressing concerns related to information security and privacy.

Several countries in Europe and North America are also considering restrictions on the use of ChatGPT due to data privacy concerns, with Italy being the first to ban it.

Altman emphasized Japan's potential in contributing to AI growth, particularly in sectors where the country has a leading role, such as anime. However, there is a growing awareness of the need for regulations to manage the risks associated with AI technologies and ensure their responsible use for the benefit of society.

Check out more news and information on Artificial Intelligence in Science Times.