年份 | 2017 |
学科 | 数学 Mathematics |
国家/州 | United States of America |
An Exploration in Textual Analysis
The internet’s expansion has increased the data available to researchers, allowing analysis to make insights into human behavior. This project statistically analyzes trends in computer science by studying the ArXiv archive’s CS articles between 1989 and 2016. Analysis show that documents’ meaningful words have a quantitative significance that a machine can extract without understanding the text.
Measurements were taken from a sample of ~8,600 documents. Their words were positively weighted by their normalized frequency and negatively weighted by either their presence in other documents (TF-IDF) or by their probability to appear in the body of a web page, in two separate measurements, to reduce the noise in plain word frequencies from “structure words” like “the” and “a.” The observed probabilities of words with large weights were graphed to show their popularity during each year. Finally, vector representations of the words were used to find contextually similar words and to examine how changes in popularity may be caused by shifts in terminology.
The secondary word relevance measurement was found comparable to TF-IDF, an algorithm used by search engines to retrieve documents: 54.9% of the 30 highest weighted words (secondary measure) are also in the 30 highest TFIDF weighted words. A significant design challenge was testing the integrity of measurements as there is little control data; this was solved by a secondary dataset of words’ probability to appear in web pages. In both measurements, words with large weights are hand-verifiably “important” to documents that they appear in, so the results show that words which are meaningful to documents have an identifiable quantitative significance.
英特尔国际科学与工程大奖赛,简称 "ISEF",由美国 Society for Science and the Public(科学和公共服务协会)主办,英特尔公司冠名赞助,是全球规模最大、等级最高的中学生的科研科创赛事。ISEF 的学术活动学科包括了所有数学、自然科学、工程的全部领域和部分社会科学。ISEF 素有全球青少年科学学术活动的“世界杯”之美誉,旨在鼓励学生团队协作,开拓创新,长期专一深入地研究自己感兴趣的课题。
The study of the measurement, properties, and relationships of quantities and sets, using numbers and symbols. The deductive study of numbers, geometry, and various abstract constructs, or structures.
Algebra (ALB): The study of algebraic operations and/or relations and the structures which arise from them. An example is given by (systems of) equations which involve polynomial functions of one or more variables.
Analysis (ANL): The study of infinitesimal processes in mathematics, typically involving the concept of a limit. This begins with differential and integral calculus, for functions of one or several variables, and includes differential equations.
Combinatorics, Graph Theory, and Game Theory (CGG): The study of combinatorial structures in mathematics, such as finite sets, graphs, and games, often with a view toward classification and/or enumeration.
Geometry and Topology (GEO): The study of the shape, size, and other properties of figures and spaces. Includes such subjects as Euclidean geometry, non-Euclidean geometries (spherical, hyperbolic, Riemannian, Lorentzian), and knot theory (classification of knots in 3-space).
Number Theory (NUM): The study of the arithmetic properties of integers and related topics such as cryptography.
Probability and Statistics (PRO): Mathematical study of random phenomena and the study of statistical tools used to analyze and interpret data.
Other (OTH): Studies that cannot be assigned to one of the above subcategories. If the project involves multiple subcategories, the principal subcategory should be chosen instead of Other.
© 2024. All Rights Reserved. 沪ICP备2023009024号-1