{ "id": "2310.15318", "version": "v1", "published": "2023-10-23T19:35:57.000Z", "updated": "2023-10-23T19:35:57.000Z", "title": "HetGPT: Harnessing the Power of Prompt Tuning in Pre-Trained Heterogeneous Graph Neural Networks", "authors": [ "Yihong Ma", "Ning Yan", "Jiayu Li", "Masood Mortazavi", "Nitesh V. Chawla" ], "comment": "submitted to ACM TheWebConf 2024", "categories": [ "cs.LG", "cs.AI" ], "abstract": "Graphs have emerged as a natural choice to represent and analyze the intricate patterns and rich information of the Web, enabling applications such as online page classification and social recommendation. The prevailing \"pre-train, fine-tune\" paradigm has been widely adopted in graph machine learning tasks, particularly in scenarios with limited labeled nodes. However, this approach often exhibits a misalignment between the training objectives of pretext tasks and those of downstream tasks. This gap can result in the \"negative transfer\" problem, wherein the knowledge gained from pre-training adversely affects performance in the downstream tasks. The surge in prompt-based learning within Natural Language Processing (NLP) suggests the potential of adapting a \"pre-train, prompt\" paradigm to graphs as an alternative. However, existing graph prompting techniques are tailored to homogeneous graphs, neglecting the inherent heterogeneity of Web graphs. To bridge this gap, we propose HetGPT, a general post-training prompting framework to improve the predictive performance of pre-trained heterogeneous graph neural networks (HGNNs). The key is the design of a novel prompting function that integrates a virtual class prompt and a heterogeneous feature prompt, with the aim to reformulate downstream tasks to mirror pretext tasks. Moreover, HetGPT introduces a multi-view neighborhood aggregation mechanism, capturing the complex neighborhood structure in heterogeneous graphs. Extensive experiments on three benchmark datasets demonstrate HetGPT's capability to enhance the performance of state-of-the-art HGNNs on semi-supervised node classification.", "revisions": [ { "version": "v1", "updated": "2023-10-23T19:35:57.000Z" } ], "analyses": { "keywords": [ "pre-trained heterogeneous graph neural networks", "downstream tasks", "prompt tuning", "benchmark datasets demonstrate hetgpts capability" ], "note": { "typesetting": "TeX", "pages": 0, "language": "en", "license": "arXiv", "status": "editable" } } }